Merge branch '345803-document-sidekiq-health-checks' into 'master'

Add optional instructions on how to enable Sidekiq health checks

See merge request gitlab-org/gitlab!76932

doc/administration/package_information/defaults.md

@@ -15,7 +15,7 @@ See the table below for the list of ports that the Omnibus GitLab assigns
 by default:
 
 |      Component       | On by default | Communicates via | Alternative |              Connection port               |
-|:-------------------:|:-------------:|:----------------:|:-----------:|:------------------------------------------:|
+|:--------------------:|:-------------:|:----------------:|:-----------:|:------------------------------------------:|
 |     GitLab Rails     |      Yes      |       Port       |      X      |                 80 or 443                  |
 |     GitLab Shell     |      Yes      |       Port       |      X      |                     22                     |
 |      PostgreSQL      |      Yes      |      Socket      | Port (5432) |                     X                      |
@@ -30,6 +30,7 @@ by default:
 |  PgBouncer exporter  |      No       |       Port       |      X      |                    9188                    |
 |   GitLab Exporter    |      Yes      |       Port       |      X      |                    9168                    |
 |   Sidekiq exporter   |      Yes      |       Port       |      X      |                    8082                    |
+| Sidekiq health check |      No       |       Port       |      X      |                    8092[^Sidekiq-health] |
 |    Puma exporter     |      No       |       Port       |      X      |                    8083                    |
 |    Geo PostgreSQL    |      No       |      Socket      | Port (5431) |                     X                      |
 |    Redis Sentinel    |      No       |       Port       |      X      |                   26379                    |
@@ -70,3 +71,5 @@ NOTE:
 In some cases, the GitLab Registry will be automatically enabled by default. Please see [our documentation](../packages/container_registry.md) for more details
 
  [^Consul-notes]: If using additional Consul functionality, more ports may need to be opened. See the [official documentation](https://www.consul.io/docs/install/ports#ports-table) for the list.
+ 
+ [^Sidekiq-health]: If Sidekiq health check settings are not set, they will default to the Sidekiq metrics exporter settings. This default is deprecated and is set to be removed in [GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/347509).

doc/administration/sidekiq.md

@@ -21,8 +21,6 @@ you want using steps 1 and 2 from the GitLab downloads page.
 1. Generate the Sidekiq configuration:
 
    ```ruby
-   sidekiq['listen_address'] = "10.10.1.48"
-
    ## Optional: Enable extra Sidekiq processes
    sidekiq_cluster['enable'] = true
    sidekiq['queue_groups'] = [
@@ -128,6 +126,34 @@ you want using steps 1 and 2 from the GitLab downloads page.
    external_url 'https://gitlab.example.com'
    ```
 
+1. (Optional) If you want to collect Sidekiq metrics, enable the Sidekiq metrics server.
+   To make metrics available from `localhost:8082/metrics`, set the following values:
+
+   ```ruby
+   sidekiq['metrics_enabled'] = true
+   sidekiq['listen_address'] = "localhost"
+   sidekiq['listen_port'] = "8082"
+   
+   # Optionally log all the metrics server logs to log/sidekiq_exporter.log
+   sidekiq['exporter_log_enabled'] = true
+   ```
+
+1. (Optional) If you use health check probes to observe Sidekiq,
+   set a separate port for health checks.
+   Configuring health checks is only necessary if there is something that actually probes them.
+   For more information about health checks, see the [Sidekiq health check page](sidekiq_health_check.md).
+   Enable health checks for Sidekiq:
+
+    ```ruby
+    sidekiq['health_checks_enabled'] = true
+    sidekiq['health_checks_listen_address'] = "localhost"
+    sidekiq['health_checks_listen_port'] = "8092"
+   ```
+
+   NOTE:
+   If health check settings are not set, they will default to the metrics exporter settings.
+   This default is deprecated and is set to be removed in [GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/347509). 
+
 1. Run `gitlab-ctl reconfigure`.
 
 You will need to restart the Sidekiq nodes after an update has occurred and database
@@ -196,7 +222,13 @@ gitlab_rails['auto_migrate'] = false
 #######################################
 ###      Sidekiq configuration      ###
 #######################################
-sidekiq['listen_address'] = "10.10.1.48"
+sidekiq['metrics_enabled'] = true
+sidekiq['exporter_log_enabled'] = false
+sidekiq['listen_port'] = "8082"
+
+sidekiq['health_checks_enabled'] = true
+sidekiq['health_checks_listen_address'] = "localhost"
+sidekiq['health_checks_listen_port'] = "8092"
 
 #######################################
 ###     Monitoring configuration    ###
@@ -230,3 +262,4 @@ Related Sidekiq configuration:
 1. [Extra Sidekiq processes](operations/extra_sidekiq_processes.md)
 1. [Extra Sidekiq routing](operations/extra_sidekiq_routing.md)
 1. [Using the GitLab-Sidekiq chart](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/)
+1. [Sidekiq health checks](sidekiq_health_check.md)

doc/administration/sidekiq_health_check.md

+---
+stage: Enablement
+group: Distribution
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Sidekiq Health Check **(FREE SELF)**
+
+GitLab provides liveness and readiness probes to indicate service health and
+reachability to the Sidekiq cluster. These endpoints 
+[can be provided to schedulers like Kubernetes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/)
+to hold traffic until the system is ready or restart the container as needed.
+
+The health check server can be set up when [configuring Sidekiq](sidekiq.md).
+
+## Readiness
+
+The readiness probe checks whether the Sidekiq workers are ready to process jobs.
+
+```plaintext
+GET /readiness
+```
+
+Assuming you set up Sidekiq's address and port to be `localhost` and `8092` respectively,
+here's an example request:
+
+```shell
+curl "http://localhost:8092/readiness"
+```
+
+On success, the endpoint returns a `200` HTTP status code, and a response like the following:
+
+```json
+{
+   "status": "ok"
+}
+```
+
+## Liveness
+
+Checks whether the Sidekiq cluster is running.
+
+```plaintext
+GET /liveness
+```
+
+Assuming you set up Sidekiq's address and port to be `localhost` and `8092` respectively,
+here's an example request:
+
+```shell
+curl "http://localhost:8092/liveness"
+```
+
+On success, the endpoint returns a `200` HTTP status code, and a response like the following:
+
+```json
+{
+   "status": "ok"
+}
+```

doc/user/admin_area/monitoring/health_check.md

@@ -131,6 +131,10 @@ On failure, the endpoint returns a `503` HTTP status code.
 
 This check is being exempt from Rack Attack.
 
+## Sidekiq
+
+Learn how to configure the [Sidekiq health checks](../../../administration/sidekiq_health_check.md).
+
 <!-- ## Troubleshooting
 
 Include any troubleshooting steps that you can foresee. If you know beforehand what issues