Merge branch 'docs-aqualls-split-metrics-page' into 'master'

Split and clean up Metrics page

See merge request gitlab-org/gitlab!36886

doc/administration/monitoring/gitlab_self_monitoring_project/index.md

@@ -73,11 +73,11 @@ you should
 
 ## Taking action on Prometheus alerts **(ULTIMATE)**
 
-You can [add a webhook](../../../operations/metrics/index.md#external-prometheus-instances)
+You can [add a webhook](../../../operations/metrics/alerts.md#external-prometheus-instances)
 to the Prometheus configuration in order for GitLab to receive notifications of any alerts.
 
 Once the webhook is setup, you can
-[take action on incoming alerts](../../../operations/metrics/index.md#taking-action-on-incidents-ultimate).
+[take action on incoming alerts](../../../operations/metrics/alerts.md#trigger-actions-from-alerts-ultimate).
 
 ## Adding custom metrics to the self monitoring project
 

doc/operations/metrics/alerts.md

+---
+stage: Monitor
+group: APM
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
+---
+
+# Set up alerts for Prometheus metrics
+
+After [configuring metrics for your CI/CD environment](index.md), you can set up
+alerting for Prometheus metrics depending on the location of your instances, and
+[trigger actions from alerts](#trigger-actions-from-alerts-ultimate) to notify
+your team when environment performance falls outside of the boundaries you set.
+
+## Managed Prometheus instances
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/6590) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.2 for [custom metrics](index.md#adding-custom-metrics), and 11.3 for [library metrics](../../user/project/integrations/prometheus_library/metrics.md).
+
+For managed Prometheus instances using auto configuration, you can
+[configure alerts for metrics](index.md#adding-custom-metrics) directly in the
+[metrics dashboard](index.md). To set an alert:
+
+1. In your project, navigate to **{cloud-gear}** **Operations > Metrics**,
+1. Identify the metric you want to create the alert for, and click the
+   **ellipsis** **{ellipsis_v}** icon in the top right corner of the metric.
+1. Choose **Alerts**.
+1. Set threshold and operator.
+1. Click **Add** to save and activate the alert.
+
+![Adding an alert](../../user/project/integrations/img/prometheus_alert.png)
+
+To remove the alert, click back on the alert icon for the desired metric, and click **Delete**.
+
+## External Prometheus instances
+
+>- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9258) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.8.
+>- [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/42640) to [GitLab Core](https://about.gitlab.com/pricing/) in 12.10.
+
+For manually configured Prometheus servers, GitLab provides a notify endpoint for
+use with Prometheus webhooks. If you have manual configuration enabled, an
+**Alerts** section is added to **{settings}** **Settings > Integrations > Prometheus**.
+This section contains the **URL** and **Authorization Key** you will need. The
+**Reset Key** button will invalidate the key and generate a new one.
+
+![Prometheus service configuration of Alerts](../../user/project/integrations/img/prometheus_service_alerts.png)
+
+To send GitLab alert notifications, copy the **URL** and **Authorization Key** into the
+[`webhook_configs`](https://prometheus.io/docs/alerting/latest/configuration/#webhook_config)
+section of your Prometheus Alertmanager configuration:
+
+```yaml
+receivers:
+  name: gitlab
+  webhook_configs:
+    - http_config:
+        bearer_token: 9e1cbfcd546896a9ea8be557caf13a76
+      send_resolved: true
+      url: http://192.168.178.31:3001/root/manual_prometheus/prometheus/alerts/notify.json
+  ...
+```
+
+For GitLab to associate your alerts with an [environment](../../ci/environments/index.md),
+you must configure a `gitlab_environment_name` label on the alerts you set up in
+Prometheus. The value of this should match the name of your environment in GitLab.
+
+NOTE: **Note:**
+In GitLab versions 13.1 and greater, you can configure your manually configured
+Prometheus server to use the
+[Generic alerts integration](../../user/project/integrations/generic_alerts.md).
+
+## Trigger actions from alerts **(ULTIMATE)**
+
+>- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/4925) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.11.
+>- [From GitLab Ultimate 12.5](https://gitlab.com/gitlab-org/gitlab/-/issues/13401), when GitLab receives a recovery alert, it will automatically close the associated issue.
+
+Alerts can be used to trigger actions, like opening an issue automatically
+(disabled by default since `13.1`). To configure the actions:
+
+1. Navigate to your project's **{settings}** **Settings > Operations > Incidents**.
+1. Enable the option to create issues.
+1. Choose the [issue template](../../user/project/description_templates.md) to create the issue from.
+1. Optionally, select whether to send an email notification to the developers of the project.
+1. Click **Save changes**.
+
+After enabling, GitLab automatically opens an issue when an alert is triggered containing
+values extracted from [alert's payload](https://prometheus.io/docs/alerting/latest/configuration/#webhook_config):
+
+- Issue author: `GitLab Alert Bot`
+- Issue title: Extract from `annotations/title`, `annotations/summary` or `labels/alertname`
+- Alert `Summary`: A list of properties
+  - `starts_at`: Alert start time via `startsAt`
+  - `full_query`: Alert query extracted from `generatorURL`
+  - Optional list of attached annotations extracted from `annotations/*`
+- Alert [GFM](../../user/markdown.md): GitLab Flavored Markdown from `annotations/gitlab_incident_markdown`
+
+When GitLab receives a **Recovery Alert**, it closes the associated issue.
+This action is recorded as a system message on the issue indicating that it
+was closed automatically by the GitLab Alert bot.
+
+To further customize the issue, you can add labels, mentions, or any other supported
+[quick action](../../user/project/quick_actions.md) in the selected issue template,
+which applies to all incidents. To limit quick actions or other information to
+only specific types of alerts, use the `annotations/gitlab_incident_markdown` field.
+
+Since [version 12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/63373),
+GitLab tags each incident issue with the `incident` label automatically. If the label
+does not yet exist, it is also created automatically.
+
+If the metric exceeds the threshold of the alert for over 5 minutes, GitLab sends
+an email to all [Maintainers and Owners](../../user/permissions.md#project-members-permissions)
+of the project.

doc/operations/metrics/dashboards/index.md

@@ -6,32 +6,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 
 # Using the Metrics Dashboard
 
-## Select an environment
-
-The **Environment** dropdown box above the dashboard displays the list of all [environments](../index.md).
-It enables you to search as you type through all environments and select the one you're looking for.
-
-![Monitoring Dashboard Environments](../../../user/project/integrations/img/prometheus_dashboard_environments_v12_8.png)
-
-## Select a dashboard
-
-The **dashboard** dropdown box above the dashboard displays the list of all dashboards available for the project.
-It enables you to search as you type through all dashboards and select the one you're looking for.
-
-![Monitoring Dashboard select](../../../user/project/integrations/img/prometheus_dashboard_select_v_13_0.png)
-
-## Mark a dashboard as favorite
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214582) in GitLab 13.0.
-
-When viewing a dashboard, click the empty **Star dashboard** **{star-o}** button to mark a
-dashboard as a favorite. Starred dashboards display a solid star **{star}** button,
-and appear at the top of the dashboard select list.
-
-To remove dashboard from the favorites list, click the solid **Unstar Dashboard** **{star}** button.
-
-![Monitoring Dashboard favorite state toggle](../../../user/project/integrations/img/toggle_metrics_user_starred_dashboard_v13_0.png)
-
 ## Manage the metrics dashboard settings
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223204) in GitLab 13.2.
@@ -57,7 +31,7 @@ The options are:
 - [View logs](#view-logs-ultimate)
 - [Download CSV](#downloading-data-as-csv)
 - [Copy link to chart](../embed.md#embedding-gitlab-managed-kubernetes-metrics)
-- [Alerts](../index.md#setting-up-alerts-for-prometheus-metrics)
+- [Alerts](../alerts.md)
 
 ### View and edit the source file of a custom dashboard
 

doc/operations/metrics/dashboards/yaml.md

@@ -76,7 +76,7 @@ Panels in a panel group are laid out in rows consisting of two panels per row. A
 
 | Property | Type | Required | Description |
 | ------ | ------ | ------ | ------ |
-| `id` | string | no | Used for associating dashboard metrics with database records. Must be unique across dashboard configuration files. Required for [alerting](../index.md#setting-up-alerts-for-prometheus-metrics) (support not yet enabled, see [relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/27980)). |
+| `id` | string | no | Used for associating dashboard metrics with database records. Must be unique across dashboard configuration files. Required for [alerting](../alerts.md) (support not yet enabled, see [relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/27980)). |
 | `unit` | string | yes | Defines the unit of the query's return data. |
 | `label` | string | no, but highly encouraged | Defines the legend-label for the query. Should be unique within the panel's metrics. Can contain time series labels as interpolated variables. |
 | `query` | string | yes if `query_range` is not defined | Defines the Prometheus query to be used to populate the chart/panel. If defined, the `query` endpoint of the [Prometheus API](https://prometheus.io/docs/prometheus/latest/querying/api/) will be utilized. |

doc/operations/metrics/embed.md

@@ -58,7 +58,7 @@ It is also possible to embed either the default dashboard metrics or individual
 
 ## Embedding metrics based on alerts in incident issues
 
-For [GitLab-managed alerting rules](index.md#setting-up-alerts-for-prometheus-metrics), the issue will include an embedded chart for the query corresponding to the alert. The chart displays an hour of data surrounding the starting point of the incident, 30 minutes before and after.
+For [GitLab-managed alerting rules](alerts.md), the issue will include an embedded chart for the query corresponding to the alert. The chart displays an hour of data surrounding the starting point of the incident, 30 minutes before and after.
 
 For [manually configured Prometheus instances](../../user/project/integrations/prometheus.md#manual-configuration-of-prometheus), a chart corresponding to the query can be included if these requirements are met:
 

doc/user/incident_management/index.md

@@ -25,7 +25,7 @@ to create issues when alerts are triggered:
    checkbox to create an issue based on your own
    [issue templates](../project/description_templates.md#creating-issue-templates).
    For more information, see
-   [Taking Action on Incidents](../../operations/metrics/index.md#taking-action-on-incidents-ultimate) **(ULTIMATE)**.
+   [Trigger actions from alerts](../../operations/metrics/alerts.md#trigger-actions-from-alerts-ultimate) **(ULTIMATE)**.
 1. To create issues from alerts, select the template in the **Issue Template**
    select box.
 1. To send [separate email notifications](#notify-developers-of-alerts) to users
@@ -36,7 +36,7 @@ to create issues when alerts are triggered:
 Appropriately configured alerts include an
 [embedded chart](../../operations/metrics/embed.md#embedding-metrics-based-on-alerts-in-incident-issues)
 for the query corresponding to the alert. You can also configure GitLab to
-[close issues](../../operations/metrics/index.md#taking-action-on-incidents-ultimate)
+[close issues](../../operations/metrics/alerts.md#trigger-actions-from-alerts-ultimate)
 when you receive notification that the alert is resolved.
 
 ### Notify developers of alerts
@@ -53,8 +53,8 @@ To send separate email notifications to users with
 
 You can set up Prometheus alerts in:
 
-- [GitLab-managed Prometheus](../../operations/metrics/index.md#setting-up-alerts-for-prometheus-metrics) installations.
-- [Self-managed Prometheus](../../operations/metrics/index.md#external-prometheus-instances) installations.
+- [GitLab-managed Prometheus](../../operations/metrics/alerts.md) installations.
+- [Self-managed Prometheus](../../operations/metrics/alerts.md#external-prometheus-instances) installations.
 
 Prometheus alerts are created by the special Alert Bot user. You can't remove this
 user, but it does not count toward your license limit.

doc/user/project/integrations/generic_alerts.md

@@ -19,7 +19,9 @@ The entire payload will be posted in the issue discussion as a comment
 authored by the GitLab Alert Bot.
 
 NOTE: **Note:**
-In GitLab versions 13.1 and greater, you can configure [External Prometheus instances](../../../operations/metrics/index.md#external-prometheus-instances) to use this endpoint.
+In GitLab versions 13.1 and greater, you can configure
+[External Prometheus instances](../../../operations/metrics/alerts.md#external-prometheus-instances)
+to use this endpoint.
 
 ## Setting up generic alerts
 

doc/user/project/integrations/prometheus.md

@@ -19,7 +19,8 @@ There are two ways to set up Prometheus integration, depending on where your app
 - For deployments on Kubernetes, GitLab can automatically [deploy and manage Prometheus](#managed-prometheus-on-kubernetes).
 - For other deployment targets, simply [specify the Prometheus server](#manual-configuration-of-prometheus).
 
-Once enabled, GitLab will automatically detect metrics from known services in the [metric library](prometheus_library/index.md). You can also [add your own metrics](../../../operations/metrics/index.md#adding-custom-metrics).
+Once enabled, GitLab will automatically detect metrics from known services in the [metric library](prometheus_library/index.md). You can also [add your own metrics](../../../operations/metrics/index.md#adding-custom-metrics) and create
+[custom dashboards](../../../operations/metrics/dashboards/index.md).
 
 ## Enabling Prometheus Integration
 
@@ -43,25 +44,6 @@ Once you have a connected Kubernetes cluster, deploying a managed Prometheus is
 
 ![Managed Prometheus Deploy](img/prometheus_deploy.png)
 
-#### Getting metrics to display on the Metrics Dashboard
-
-After completing the steps above, you will also need deployments in order to view the
-**Operations > Metrics** page. Setting up [Auto DevOps](../../../topics/autodevops/index.md)
-will help you to quickly create a deployment:
-
-1. Navigate to your project's **Operations > Kubernetes** page, and ensure that,
-   in addition to "Prometheus", you also have "Runner" and "Ingress"
-   installed. Once "Ingress" is installed, copy its endpoint.
-1. Navigate to your project's **Settings > CI/CD** page. In the Auto DevOps section,
-   select a deployment strategy and save your changes.
-1. On the same page, in the Variables section, add a variable named `KUBE_INGRESS_BASE_DOMAIN`
-   with the value of the Ingress endpoint you have copied in the previous step. Leave the type
-   as "Variable".
-1. Navigate to your project's **CI/CD > Pipelines** page, and run a pipeline on any branch.
-1. When the pipeline has run successfully, graphs will be available on the **Operations > Metrics** page.
-
-![Monitoring Dashboard](img/prometheus_monitoring_dashboard_v13_1.png)
-
 #### About managed Prometheus deployments
 
 Prometheus is deployed into the `gitlab-managed-apps` namespace, using the [official Helm chart](https://github.com/helm/charts/tree/master/stable/prometheus). Prometheus is only accessible within the cluster, with GitLab communicating through the [Kubernetes API](https://kubernetes.io/docs/concepts/overview/kubernetes-api/).

doc/user/project/operations/alert_management.md

@@ -65,16 +65,16 @@ When GitLab-managed Prometheus is installed, the [Alerts list](#alert-management
 is also enabled.
 
 To populate the alerts with data, see
-[GitLab-Managed Prometheus instances](../../../operations/metrics/index.md#managed-prometheus-instances).
+[GitLab-Managed Prometheus instances](../../../operations/metrics/alerts.md#managed-prometheus-instances).
 
 ### Enable external Prometheus alerts
 
 You can configure an externally-managed Prometheus instance to send alerts
-to GitLab. To set up this configuration, see the [configuring Prometheus](../../../operations/metrics/index.md#external-prometheus-instances) documentation. Activating the external Prometheus
+to GitLab. To set up this configuration, see the [configuring Prometheus](../../../operations/metrics/alerts.md#external-prometheus-instances) documentation. Activating the external Prometheus
 configuration also enables the [Alerts list](#alert-management-list).
 
 To populate the alerts with data, see
-[External Prometheus instances](../../../operations/metrics/index.md#external-prometheus-instances).
+[External Prometheus instances](../../../operations/metrics/alerts.md#external-prometheus-instances).
 
 ## Alert Management severity
 
@@ -240,14 +240,14 @@ To view the metrics for an alert:
 
 For GitLab-managed Prometheus instances, metrics data is automatically available
 for the alert, making it easy to see surrounding behavior. See
-[Managed Prometheus instances](../../../operations/metrics/index.md#managed-prometheus-instances)
+[Managed Prometheus instances](../../../operations/metrics/alerts.md#managed-prometheus-instances)
 for information on setting up alerts.
 
 For externally-managed Prometheus instances, you can configure your alerting rules to
 display a chart in the alert. See
 [Embedding metrics based on alerts in incident issues](../../../operations/metrics/embed.md#embedding-metrics-based-on-alerts-in-incident-issues)
 for information on how to appropriately configure your alerting rules. See
-[External Prometheus instances](../../../operations/metrics/index.md#external-prometheus-instances)
+[External Prometheus instances](../../../operations/metrics/alerts.md#external-prometheus-instances)
 for information on setting up alerts for your self-managed Prometheus instance.
 
 ## Use cases for assigning alerts