| [GitLab Cycle Analytics](user/project/cycle_analytics.md) | Measure the time it takes to go from an [idea to production](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#from-idea-to-production-with-gitlab) for each project you have. |
| [GitLab Value Stream Analytics](user/project/cycle_analytics.md) | Measure the time it takes to go from an [idea to production](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#from-idea-to-production-with-gitlab) for each project you have. |
| [Instance Statistics](user/instance_statistics/index.md) | Discover statistics on how many GitLab features you use and user activity. |
Cycle analytics calculates the time between two arbitrary events recorded on domain objects and provides aggregated statistics about the duration.
## Stage
During development, events occur that move issues and merge requests through different stages of progress until they are considered finished. These stages can be expressed with the `Stage` model.
Example stage:
- Name: Development
- Start event: Issue created
- End event: Issue first mentioned in commit
- Parent: `Group: gitlab-org`
### Events
Events are the smallest building blocks of the cycle analytics feature. A stage consists of two events:
- Start
- End
These events play a key role in the duration calculation.
To make the duration calculation flexible, each `Event` is implemented as a separate class. They're responsible for defining a timestamp expression that will be used in the calculation query.
#### Implementing an `Event` class
There are a few methods that are required to be implemented, the `StageEvent` base class describes them in great detail. The most important ones are:
-`object_type`
-`timestamp_projection`
The `object_type` method defines which domain object will be queried for the calculation. Currently two models are allowed:
-`Issue`
-`MergeRequest`
For the duration calculation the `timestamp_projection` method will be used.
```ruby
deftimestamp_projection
# your timestamp expression comes here
end
# event will use the issue creation time in the duration calculation
deftimestamp_projection
Issue.arel_table[:created_at]
end
```
NOTE: **Note:**
More complex expressions are also possible (e.g. using `COALESCE`). Look at the existing event classes for examples.
In some cases, defining the `timestamp_projection` method is not enough. The calculation query should know which table contains the timestamp expression. Each `Event` class is responsible for making modifications to the calculation query to make the `timestamp_projection` work. This usually means joining an additional table.
Example for joining the `issue_metrics` table and using the `first_mentioned_in_commit_at` column as the timestamp expression:
# in this case the query attribute will be based on the Issue model: `Issue.where(...)`
query.joins(:metrics)
end
```
### Validating start and end events
Some start/end event pairs are not "compatible" with each other. For example:
- "Issue created" to "Merge Request created": The event classes are defined on different domain models, the `object_type` method is different.
- "Issue closed" to "Issue created": Issue must be created first before it can be closed.
- "Issue closed" to "Issue closed": Duration is always 0.
The `StageEvents` module describes the allowed `start_event` and `end_event` pairings (`PAIRING_RULES` constant). If a new event is added, it needs to be registered in this module.
To add a new event:
1. Add an entry in `ENUM_MAPPING` with a unique number, it'll be used in the `Stage` model as `enum`.
1. Define which events are compatible with the event in the `PAIRING_RULES` hash.
Teams and organizations might define their own way of building software, thus stages can be completely different. For each stage, a parent object needs to be defined.
Currently supported parents:
-`Project`
-`Group`
#### How parent relationship it work
1. User navigates to the cycle analytics page.
1. User selects a group.
1. Backend loads the defined stages for the selected group.
1. Additions and modifications to the stages will be persisted within the selected group only.
### Default stages
The [original implementation](https://gitlab.com/gitlab-org/gitlab/issues/847) of cycle analytics defined 7 stages. These stages are always available for each parent, however altering these stages is not possible.
To make things efficient and reduce the number of records created, the default stages are expressed as in-memory objects (not persisted). When the user creates a custom stage for the first time, all the stages will be persisted. This behaviour is implemented in the cycle analytics service objects.
The reason for this was that we'd like to add the abilities to hide and order stages later on.
## Data Collector
`DataCollector` is the central point where the data will be queried from the database. The class always operates on a single stage and consists of the following components:
-`BaseQueryBuilder`:
- Responsible for composing the initial query.
- Deals with `Stage` specific configuration: events and their query customizations.
- Parameters coming from the UI: date ranges.
-`Median`: Calculates the median duration for a stage using the query from `BaseQueryBuilder`.
-`RecordsFetcher`: Loads relevant records for a stage using the query from `BaseQueryBuilder` and specific `Finder` classes to apply visibility rules.
-`DataForDurationChart`: Loads calculated durations with the finish time (end event timestamp) for the scatterplot chart.
For a new calculation or a query, implement it as a new method call in the `DataCollector` class.
- Rails Controller (`Analytics::CycleAnalytics` module): Cycle analytics exposes its data via JSON endpoints, implemented within the `analytics` workspace. Configuring the stages are also implements JSON endpoints (CRUD).
- Services (`Analytics::CycleAnalytics` module): All `Stage` related actions will be delegated to respective service objects.
- Models (`Analytics::CycleAnalytics` module): Models are used to persist the `Stage` objects `ProjectStage` and `GroupStage`.
- Responsible for composing queries and define feature specific busines logic.
-`DataCollector`, `Event`, `StageEvents`, etc.
## Testing
Since we have a lots of events and possible pairings, testing each pairing is not possible. The rule is to have at least one test case using an `Event` class.
Writing a test case for a stage using a new `Event` can be challenging since data must be created for both events. To make this a bit simpler, each test case must be implemented in the `data_collector_spec.rb` where the stage is tested through the `DataCollector`. Each test case will be turned into multiple tests, covering the following cases:
- Different parents: `Group` or `Project`
- Different calculations: `Median`, `RecordsFetcher` or `DataForDurationChart`
This document was moved to [another location](value_stream_analytics.md)
Value stream analytics calculates the time between two arbitrary events recorded on domain objects and provides aggregated statistics about the duration.
For information on how to configure Value Stream Analytics in GitLab, see our [analytics documentation](../user/analytics/value_stream_analytics.md).
## Stage
During development, events occur that move issues and merge requests through different stages of progress until they are considered finished. These stages can be expressed with the `Stage` model.
Example stage:
- Name: Development
- Start event: Issue created
- End event: Issue first mentioned in commit
- Parent: `Group: gitlab-org`
### Events
Events are the smallest building blocks of the value stream analytics feature. A stage consists of two events:
- Start
- End
These events play a key role in the duration calculation.
To make the duration calculation flexible, each `Event` is implemented as a separate class. They're responsible for defining a timestamp expression that will be used in the calculation query.
#### Implementing an `Event` class
There are a few methods that are required to be implemented, the `StageEvent` base class describes them in great detail. The most important ones are:
-`object_type`
-`timestamp_projection`
The `object_type` method defines which domain object will be queried for the calculation. Currently two models are allowed:
-`Issue`
-`MergeRequest`
For the duration calculation the `timestamp_projection` method will be used.
```ruby
deftimestamp_projection
# your timestamp expression comes here
end
# event will use the issue creation time in the duration calculation
deftimestamp_projection
Issue.arel_table[:created_at]
end
```
NOTE: **Note:**
More complex expressions are also possible (e.g. using `COALESCE`). Look at the existing event classes for examples.
In some cases, defining the `timestamp_projection` method is not enough. The calculation query should know which table contains the timestamp expression. Each `Event` class is responsible for making modifications to the calculation query to make the `timestamp_projection` work. This usually means joining an additional table.
Example for joining the `issue_metrics` table and using the `first_mentioned_in_commit_at` column as the timestamp expression:
# in this case the query attribute will be based on the Issue model: `Issue.where(...)`
query.joins(:metrics)
end
```
### Validating start and end events
Some start/end event pairs are not "compatible" with each other. For example:
- "Issue created" to "Merge Request created": The event classes are defined on different domain models, the `object_type` method is different.
- "Issue closed" to "Issue created": Issue must be created first before it can be closed.
- "Issue closed" to "Issue closed": Duration is always 0.
The `StageEvents` module describes the allowed `start_event` and `end_event` pairings (`PAIRING_RULES` constant). If a new event is added, it needs to be registered in this module.
To add a new event:
1. Add an entry in `ENUM_MAPPING` with a unique number, it'll be used in the `Stage` model as `enum`.
1. Define which events are compatible with the event in the `PAIRING_RULES` hash.
Teams and organizations might define their own way of building software, thus stages can be completely different. For each stage, a parent object needs to be defined.
Currently supported parents:
-`Project`
-`Group`
#### How parent relationship it work
1. User navigates to the value stream analytics page.
1. User selects a group.
1. Backend loads the defined stages for the selected group.
1. Additions and modifications to the stages will be persisted within the selected group only.
### Default stages
The [original implementation](https://gitlab.com/gitlab-org/gitlab/issues/847) of value stream analytics defined 7 stages. These stages are always available for each parent, however altering these stages is not possible.
To make things efficient and reduce the number of records created, the default stages are expressed as in-memory objects (not persisted). When the user creates a custom stage for the first time, all the stages will be persisted. This behaviour is implemented in the value stream analytics service objects.
The reason for this was that we'd like to add the abilities to hide and order stages later on.
## Data Collector
`DataCollector` is the central point where the data will be queried from the database. The class always operates on a single stage and consists of the following components:
-`BaseQueryBuilder`:
- Responsible for composing the initial query.
- Deals with `Stage` specific configuration: events and their query customizations.
- Parameters coming from the UI: date ranges.
-`Median`: Calculates the median duration for a stage using the query from `BaseQueryBuilder`.
-`RecordsFetcher`: Loads relevant records for a stage using the query from `BaseQueryBuilder` and specific `Finder` classes to apply visibility rules.
-`DataForDurationChart`: Loads calculated durations with the finish time (end event timestamp) for the scatterplot chart.
For a new calculation or a query, implement it as a new method call in the `DataCollector` class.
- Rails Controller (`Analytics::CycleAnalytics` module): Value stream analytics exposes its data via JSON endpoints, implemented within the `analytics` workspace. Configuring the stages are also implements JSON endpoints (CRUD).
- Services (`Analytics::CycleAnalytics` module): All `Stage` related actions will be delegated to respective service objects.
- Models (`Analytics::CycleAnalytics` module): Models are used to persist the `Stage` objects `ProjectStage` and `GroupStage`.
- Responsible for composing queries and define feature specific busines logic.
-`DataCollector`, `Event`, `StageEvents`, etc.
## Testing
Since we have a lots of events and possible pairings, testing each pairing is not possible. The rule is to have at least one test case using an `Event` class.
Writing a test case for a stage using a new `Event` can be challenging since data must be created for both events. To make this a bit simpler, each test case must be implemented in the `data_collector_spec.rb` where the stage is tested through the `DataCollector`. Each test case will be turned into multiple tests, covering the following cases:
- Different parents: `Group` or `Project`
- Different calculations: `Median`, `RecordsFetcher` or `DataForDurationChart`
A web application firewall (or WAF) filters, monitors, and blocks HTTP traffic to
and from a web application. By inspecting HTTP traffic, it can prevent attacks
stemming from web application security flaws. It can be used to detect SQL injection,
Cross-Site Scripting (XSS), Remote File Inclusion, Security Misconfigurations, and
much more.
## Overview
GitLab provides a WAF out of the box after Ingress is deployed.
All you need to do is deploy your application along with a service
and Ingress resource.
In GitLab's [Ingress](../../user/clusters/applications.md#ingress) deployment, the [ModSecurity](https://modsecurity.org/) module is loaded
into Ingress-NGINX by default and monitors the traffic going to the
applications which have an Ingress.
The ModSecurity module runs with the [OWASP Core Rule Set (CRS)](https://coreruleset.org/) by default. The OWASP CRS will detect and log a wide range of common attacks.
NOTE: **Note**
The WAF is deployed in "Detection-only mode" by default and will only log attack
attempts.
## Requirements
The Web Application Firewall requires:
-**Kubernetes**
To enable the WAF, you need:
- Kubernetes 1.12+.
- A load balancer. You can use NGINX-Ingress by deploying it to your
Kubernetes cluster by either:
- Using the [`nginx-ingress` Helm chart](https://github.com/helm/charts/tree/master/stable/nginx-ingress).
- Installing the [Ingress GitLab Managed App](../../user/clusters/applications.md#ingress) with WAF enabled.
-**Configured Kubernetes objects**
To use the WAF on an application, you need to deploy the following Kubernetes resources:
# Getting started with the Web Application Firewall
This is a step-by-step guide that will help you use GitLab's [Web Application Firewall](index.md) after
deploying a project hosted on GitLab.com to Google Kubernetes Engine using [Auto DevOps](../autodevops/index.md).
We will use GitLab's native Kubernetes integration, so you will not need
to create a Kubernetes cluster manually using the Google Cloud Platform console.
We will create and deploy a simple application that we create from a GitLab template.
These instructions will also work for a self-hosted GitLab instance. However, you will
need to ensure your own [Runners are configured](../../ci/runners/README.md) and
[Google OAuth is enabled](../../integration/google.md).
**Note**: GitLab's Web Application Firewall is deployed with [Ingress](../../user/clusters/applications.md#Ingress),
so it will be avaliable to your applications no matter how you deploy them to Kubernetes.
## Enable or disable ModSecurity
ModSecurity is enabled by default on GitLab.com. You can toggle the feature flag to false by running the following command in the Rails console:
```ruby
Feature.disable(:ingress_modsecurity)
```
Once disabled, you must uninstall and reinstall your Ingress application for the changes to take effect. See the [Feature Flag](../../user/project/operations/feature_flags.md) documentation for more information.
## Configuring your Google account
Before creating and connecting your Kubernetes cluster to your GitLab project,
you need a Google Cloud Platform account. If you do not already have one,
sign up at <https://console.cloud.google.com>. You will need to either sign in with an existing
Google account (for example, one that you use to access Gmail, Drive, etc.) or create a new one.
1. To enable the required APIs and related services, follow the steps in the ["Before you begin" section of the Kubernetes Engine docs](https://cloud.google.com/kubernetes-engine/docs/quickstart#before-you-begin).
1. Make sure you have created a [billing account](https://cloud.google.com/billing/docs/how-to/manage-billing-account).
TIP: **Tip:**
Every new Google Cloud Platform (GCP) account receives [$300 in credit](https://console.cloud.google.com/freetrial),
and in partnership with Google, GitLab is able to offer an additional $200 for new GCP accounts to get started with GitLab's
Google Kubernetes Engine integration. All you have to do is [follow this link](https://cloud.google.com/partners/partnercredit/?PCN=a0n60000006Vpz4AAC) and apply for credit.
## Creating a new project from a template
We will use one of GitLab's project templates to get started. As the name suggests,
those projects provide a barebones application built on some well-known frameworks.
1. In GitLab, click the plus icon (**+**) at the top of the navigation bar and select
**New project**.
1. Go to the **Create from template** tab where you can choose for example a Ruby on
1. The last step is to provide the cluster details.
1. Give it a name, leave the environment scope as is, and choose the GCP project under which the cluster
will be created (per the instructions to [configure your Google account](#configuring-your-google-account), a project should have already been created for you).
1. Choose the [region/zone](https://cloud.google.com/compute/docs/regions-zones/) under which the cluster will be created.
1. Enter the number of nodes you want it to have.
1. Choose the [machine type](https://cloud.google.com/compute/docs/machine-types).
You can see that ModSecurity logs the suspicous behavior. By sending a request
with the `User Agent: absinthe` header, which [absinthe](https://github.com/cameronhotchkies/Absinthe), a tool for testing for SQL injections uses, we can detect that someone was
searching for vulnerabilities on our system. Detecting scanners is useful, because we
can learn if someone is trying to exploit our system.
## Conclusion
You can now see the benefits of a using a Web Application Firewall.
ModSecurity and the OWASP Core Rule Set, offer many more benefits.
> - Introduced prior to GitLab 12.3 at the project level.
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12077) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.3 at the group level.
Cycle Analytics measures the time spent to go from an
[idea to production](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#from-idea-to-production-with-gitlab)
(also known as cycle time) for each of your projects. Cycle Analytics displays the median time
spent in each stage defined in the process.
NOTE: **Note:**
Use the `cycle_analytics` feature flag to enable at the group level.
Cycle Analytics is useful in order to quickly determine the velocity of a given
project. It points to bottlenecks in the development process, enabling management
to uncover, triage, and identify the root cause of slowdowns in the software development life cycle.
Cycle Analytics is tightly coupled with the [GitLab flow](../../topics/gitlab_flow.md) and
calculates a separate median for each stage.
## Overview
Cycle Analytics is available:
- From GitLab 12.3, at the group level in the analytics workspace (top navigation bar) at
**Analytics > Cycle Analytics**. **(PREMIUM)**
In the future, multiple groups will be selectable which will effectively make this an
instance-level feature.
- At the project level via **Project > Cycle Analytics**.
There are seven stages that are tracked as part of the Cycle Analytics calculations.
-**Issue** (Tracker)
- Time to schedule an issue (by milestone or by adding it to an issue board)
-**Plan** (Board)
- Time to first commit
-**Code** (IDE)
- Time to create a merge request
-**Test** (CI)
- Time it takes GitLab CI/CD to test your code
-**Review** (Merge Request/MR)
- Time spent on code review
-**Staging** (Continuous Deployment)
- Time between merging and deploying to production
-**Total** (Total)
- Total lifecycle time. That is, the velocity of the project or team. [Previously known](https://gitlab.com/gitlab-org/gitlab/issues/38317) as **Production**.
## Date ranges
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/13216) in GitLab 12.4.
GitLab provides the ability to filter analytics based on a date range. To filter results:
1. Select a group.
1. Optionally select a project.
1. Select a date range using the available date pickers.
## How the data is measured
Cycle Analytics records cycle time and data based on the project issues with the
exception of the staging and total stages, where only data deployed to
production are measured.
Specifically, if your CI is not set up and you have not defined a `production`
or `production/*`[environment](../../ci/yaml/README.md#environment), then you will not have any
data for this stage.
Each stage of Cycle Analytics is further described in the table below.
| **Stage** | **Description** |
| --------- | --------------- |
| Issue | Measures the median time between creating an issue and taking action to solve it, by either labeling it or adding it to a milestone, whatever comes first. The label will be tracked only if it already has an [Issue Board list](../project/issue_board.md#creating-a-new-list) created for it. |
| Plan | Measures the median time between the action you took for the previous stage, and pushing the first commit to the branch. The very first commit of the branch is the one that triggers the separation between **Plan** and **Code**, and at least one of the commits in the branch needs to contain the related issue number (e.g., `#42`). If none of the commits in the branch mention the related issue number, it is not considered to the measurement time of the stage. |
| Code | Measures the median time between pushing a first commit (previous stage) and creating a merge request (MR) related to that commit. The key to keep the process tracked is to include the [issue closing pattern](../project/issues/managing_issues.md#closing-issues-automatically) to the description of the merge request (for example, `Closes #xxx`, where `xxx` is the number of the issue related to this merge request). If the issue closing pattern is not present in the merge request description, the MR is not considered to the measurement time of the stage. |
| Test | Measures the median time to run the entire pipeline for that project. It's related to the time GitLab CI takes to run every job for the commits pushed to that merge request defined in the previous stage. It is basically the start->finish time for all pipelines. |
| Review | Measures the median time taken to review the merge request that has closing issue pattern, between its creation and until it's merged. |
| Staging | Measures the median time between merging the merge request with closing issue pattern until the very first deployment to production. It's tracked by the environment set to `production` or matching `production/*` (case-sensitive, `Production` won't work) in your GitLab CI configuration. If there isn't a production environment, this is not tracked. |
| Total | The sum of all time (medians) taken to run the entire process, from issue creation to deploying the code to production. [Previously known](https://gitlab.com/gitlab-org/gitlab/issues/38317) as **Production**. |
How this works, behind the scenes:
1. Issues and merge requests are grouped together in pairs, such that for each
`<issue, merge request>` pair, the merge request has the [issue closing pattern](../project/issues/managing_issues.md#closing-issues-automatically)
for the corresponding issue. All other issues and merge requests are **not**
considered.
1. Then the `<issue, merge request>` pairs are filtered out by last XX days (specified
by the UI - default is 90 days). So it prohibits these pairs from being considered.
1. For the remaining `<issue, merge request>` pairs, we check the information that
we need for the stages, like issue creation date, merge request merge time,
etc.
To sum up, anything that doesn't follow [GitLab flow](../../workflow/gitlab_flow.md) will not be tracked and the
Cycle Analytics dashboard will not present any data for:
- Merge requests that do not close an issue.
- Issues not labeled with a label present in the Issue Board or for issues not assigned a milestone.
- Staging and production stages, if the project has no `production` or `production/*`
environment.
## Example workflow
Below is a simple fictional workflow of a single cycle that happens in a
single day passing through all seven stages. Note that if a stage does not have
a start and a stop mark, it is not measured and hence not calculated in the median
time. It is assumed that milestones are created and CI for testing and setting
environments is configured.
1. Issue is created at 09:00 (start of **Issue** stage).
1. Issue is added to a milestone at 11:00 (stop of **Issue** stage / start of
**Plan** stage).
1. Start working on the issue, create a branch locally and make one commit at
12:00.
1. Make a second commit to the branch which mentions the issue number at 12.30
(stop of **Plan** stage / start of **Code** stage).
1. Push branch and create a merge request that contains the [issue closing pattern](../project/issues/managing_issues.md#closing-issues-automatically)
in its description at 14:00 (stop of **Code** stage / start of **Test** and
**Review** stages).
1. The CI starts running your scripts defined in [`.gitlab-ci.yml`](../../ci/yaml/README.md) and
takes 5min (stop of **Test** stage).
1. Review merge request, ensure that everything is OK and merge the merge
request at 19:00. (stop of **Review** stage / start of **Staging** stage).
1. Now that the merge request is merged, a deployment to the `production`
environment starts and finishes at 19:30 (stop of **Staging** stage).
1. The cycle completes and the sum of the median times of the previous stages
is recorded to the **Total** stage. That is the time between creating an
issue and deploying its relevant merge request to production.
From the above example you can conclude the time it took each stage to complete
as long as their total time:
-**Issue**: 2h (11:00 - 09:00)
-**Plan**: 1h (12:00 - 11:00)
-**Code**: 2h (14:00 - 12:00)
-**Test**: 5min
-**Review**: 5h (19:00 - 14:00)
-**Staging**: 30min (19:30 - 19:00)
-**Total**: Since this stage measures the sum of median time of all
previous stages, we cannot calculate it if we don't know the status of the
stages before. In case this is the very first cycle that is run in the project,
then the **Total** time is 10h 30min (19:30 - 09:00)
A few notes:
- In the above example we demonstrated that it doesn't matter if your first
commit doesn't mention the issue number, you can do this later in any commit
of the branch you are working on.
- You can see that the **Test** stage is not calculated to the overall time of
the cycle since it is included in the **Review** process (every MR should be
tested).
- The example above was just **one cycle** of the seven stages. Add multiple
cycles, calculate their median time and the result is what the dashboard of
Cycle Analytics is showing.
## Days to completion chart
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/21631) in GitLab 12.6.
This chart visually depicts the total number of days it takes for cycles to be completed.
This chart uses the global page filters for displaying data based on the selected
group, projects, and timeframe. In addition, specific stages can be selected
from within the chart itself.
### Chart median line
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/36675) in GitLab 12.7.
The median line on the chart displays data that is offset by the number of days selected.
For example, if 30 days worth of data has been selected (for example, 2019-12-16 to 2020-01-15) the
median line will represent the previous 30 days worth of data (2019-11-16 to 2019-12-16)
as a metric to compare against.
### Disabling chart
This chart is enabled by default. If you have a self-managed instance, an
administrator can open a Rails console and disable it with the following command:
@@ -7,7 +7,7 @@ Track development velocity with Productivity Analytics.
For many companies, the development cycle is a blackbox and getting an estimate of how
long, on average, it takes to deliver features is an enormous endeavor.
While [Cycle Analytics](../project/cycle_analytics.md) focuses on the entire
While [Value Stream Analytics](../project/cycle_analytics.md) focuses on the entire
Software Development Life Cycle (SDLC) process, Productivity Analytics provides a way for Engineering Management to drill down in a systematic way to uncover patterns and causes for success or failure at an individual, project or group level.
Productivity can slow down for many reasons ranging from degrading code base to quickly growing teams. In order to investigate, department or team leaders can start by visualizing the time it takes for merge requests to be merged.
> - Introduced as Cycle Analytics prior to GitLab 12.3 at the project level.
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12077) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.3 at the group level.
> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/23427) from Cycle Analytics to Value Stream Analytics in GitLab 12.8.
Value Stream Analytics measures the time spent to go from an
[idea to production](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#from-idea-to-production-with-gitlab)
(also known as cycle time) for each of your projects. Value Stream Analytics displays the median time
spent in each stage defined in the process.
For information on how to contribute to the development of Value Stream Analytics, see our [contributor documentation](../../development/value_stream_analytics.md).
NOTE: **Note:**
Use the `cycle_analytics` feature flag to enable at the group level.
Value Stream Analytics is useful in order to quickly determine the velocity of a given
project. It points to bottlenecks in the development process, enabling management
to uncover, triage, and identify the root cause of slowdowns in the software development life cycle.
Value Stream Analytics is tightly coupled with the [GitLab flow](../../topics/gitlab_flow.md) and
calculates a separate median for each stage.
## Overview
Value Stream Analytics is available:
- From GitLab 12.3, at the group level in the analytics workspace (top navigation bar) at
**Analytics > Value Stream Analytics**. **(PREMIUM)**
In the future, multiple groups will be selectable which will effectively make this an
instance-level feature.
- At the project level via **Project > Value Stream Analytics**.
There are seven stages that are tracked as part of the Value Stream Analytics calculations.
-**Issue** (Tracker)
- Time to schedule an issue (by milestone or by adding it to an issue board)
-**Plan** (Board)
- Time to first commit
-**Code** (IDE)
- Time to create a merge request
-**Test** (CI)
- Time it takes GitLab CI/CD to test your code
-**Review** (Merge Request/MR)
- Time spent on code review
-**Staging** (Continuous Deployment)
- Time between merging and deploying to production
-**Total** (Total)
- Total lifecycle time. That is, the velocity of the project or team. [Previously known](https://gitlab.com/gitlab-org/gitlab/issues/38317) as **Production**.
## Date ranges
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/13216) in GitLab 12.4.
GitLab provides the ability to filter analytics based on a date range. To filter results:
1. Select a group.
1. Optionally select a project.
1. Select a date range using the available date pickers.
## How the data is measured
Value Stream Analytics records cycle time and data based on the project issues with the
exception of the staging and total stages, where only data deployed to
production are measured.
Specifically, if your CI is not set up and you have not defined a `production`
or `production/*`[environment](../../ci/yaml/README.md#environment), then you will not have any
data for this stage.
Each stage of Value Stream Analytics is further described in the table below.
| **Stage** | **Description** |
| --------- | --------------- |
| Issue | Measures the median time between creating an issue and taking action to solve it, by either labeling it or adding it to a milestone, whatever comes first. The label will be tracked only if it already has an [Issue Board list](../project/issue_board.md#creating-a-new-list) created for it. |
| Plan | Measures the median time between the action you took for the previous stage, and pushing the first commit to the branch. The very first commit of the branch is the one that triggers the separation between **Plan** and **Code**, and at least one of the commits in the branch needs to contain the related issue number (e.g., `#42`). If none of the commits in the branch mention the related issue number, it is not considered to the measurement time of the stage. |
| Code | Measures the median time between pushing a first commit (previous stage) and creating a merge request (MR) related to that commit. The key to keep the process tracked is to include the [issue closing pattern](../project/issues/managing_issues.md#closing-issues-automatically) to the description of the merge request (for example, `Closes #xxx`, where `xxx` is the number of the issue related to this merge request). If the issue closing pattern is not present in the merge request description, the MR is not considered to the measurement time of the stage. |
| Test | Measures the median time to run the entire pipeline for that project. It's related to the time GitLab CI takes to run every job for the commits pushed to that merge request defined in the previous stage. It is basically the start->finish time for all pipelines. |
| Review | Measures the median time taken to review the merge request that has closing issue pattern, between its creation and until it's merged. |
| Staging | Measures the median time between merging the merge request with closing issue pattern until the very first deployment to production. It's tracked by the environment set to `production` or matching `production/*` (case-sensitive, `Production` won't work) in your GitLab CI configuration. If there isn't a production environment, this is not tracked. |
| Total | The sum of all time (medians) taken to run the entire process, from issue creation to deploying the code to production. [Previously known](https://gitlab.com/gitlab-org/gitlab/issues/38317) as **Production**. |
How this works, behind the scenes:
1. Issues and merge requests are grouped together in pairs, such that for each
`<issue, merge request>` pair, the merge request has the [issue closing pattern](../project/issues/managing_issues.md#closing-issues-automatically)
for the corresponding issue. All other issues and merge requests are **not**
considered.
1. Then the `<issue, merge request>` pairs are filtered out by last XX days (specified
by the UI - default is 90 days). So it prohibits these pairs from being considered.
1. For the remaining `<issue, merge request>` pairs, we check the information that
we need for the stages, like issue creation date, merge request merge time,
etc.
To sum up, anything that doesn't follow [GitLab flow](../../workflow/gitlab_flow.md) will not be tracked and the
Value Stream Analytics dashboard will not present any data for:
- Merge requests that do not close an issue.
- Issues not labeled with a label present in the Issue Board or for issues not assigned a milestone.
- Staging and production stages, if the project has no `production` or `production/*`
environment.
## Example workflow
Below is a simple fictional workflow of a single cycle that happens in a
single day passing through all seven stages. Note that if a stage does not have
a start and a stop mark, it is not measured and hence not calculated in the median
time. It is assumed that milestones are created and CI for testing and setting
environments is configured.
1. Issue is created at 09:00 (start of **Issue** stage).
1. Issue is added to a milestone at 11:00 (stop of **Issue** stage / start of
**Plan** stage).
1. Start working on the issue, create a branch locally and make one commit at
12:00.
1. Make a second commit to the branch which mentions the issue number at 12.30
(stop of **Plan** stage / start of **Code** stage).
1. Push branch and create a merge request that contains the [issue closing pattern](../project/issues/managing_issues.md#closing-issues-automatically)
in its description at 14:00 (stop of **Code** stage / start of **Test** and
**Review** stages).
1. The CI starts running your scripts defined in [`.gitlab-ci.yml`](../../ci/yaml/README.md) and
takes 5min (stop of **Test** stage).
1. Review merge request, ensure that everything is OK and merge the merge
request at 19:00. (stop of **Review** stage / start of **Staging** stage).
1. Now that the merge request is merged, a deployment to the `production`
environment starts and finishes at 19:30 (stop of **Staging** stage).
1. The cycle completes and the sum of the median times of the previous stages
is recorded to the **Total** stage. That is the time between creating an
issue and deploying its relevant merge request to production.
From the above example you can conclude the time it took each stage to complete
as long as their total time:
-**Issue**: 2h (11:00 - 09:00)
-**Plan**: 1h (12:00 - 11:00)
-**Code**: 2h (14:00 - 12:00)
-**Test**: 5min
-**Review**: 5h (19:00 - 14:00)
-**Staging**: 30min (19:30 - 19:00)
-**Total**: Since this stage measures the sum of median time of all
previous stages, we cannot calculate it if we don't know the status of the
stages before. In case this is the very first cycle that is run in the project,
then the **Total** time is 10h 30min (19:30 - 09:00)
A few notes:
- In the above example we demonstrated that it doesn't matter if your first
commit doesn't mention the issue number, you can do this later in any commit
of the branch you are working on.
- You can see that the **Test** stage is not calculated to the overall time of
the cycle since it is included in the **Review** process (every MR should be
tested).
- The example above was just **one cycle** of the seven stages. Add multiple
cycles, calculate their median time and the result is what the dashboard of
Value Stream Analytics is showing.
## Days to completion chart
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/21631) in GitLab 12.6.
This chart visually depicts the total number of days it takes for cycles to be completed.
This chart uses the global page filters for displaying data based on the selected
group, projects, and timeframe. In addition, specific stages can be selected
from within the chart itself.
### Chart median line
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/36675) in GitLab 12.7.
The median line on the chart displays data that is offset by the number of days selected.
For example, if 30 days worth of data has been selected (for example, 2019-12-16 to 2020-01-15) the
median line will represent the previous 30 days worth of data (2019-11-16 to 2019-12-16)
as a metric to compare against.
### Disabling chart
This chart is enabled by default. If you have a self-managed instance, an
administrator can open a Rails console and disable it with the following command:
A Web Application Firewall (WAF) is able to examine traffic being sent/received
and can block malicious traffic before it reaches your application. The benefits
of a WAF are:
- Real-time security monitoring for your application
- Logging of all your HTTP traffic to the application
- Access control for your application
- Highly configurable logging and blocking rules
Out of the box, GitLab provides you with a WAF known as [`ModSecurity`](https://www.modsecurity.org/)
Modsecurity is a toolkit for real-time web application monitoring, logging,
and access control. With GitLab's offering, the [OWASP's Core Rule Set](https://www.modsecurity.org/CRS/Documentation/), which provides generic attack detection capabilities,
is automatically applied.
ModSecurity is a toolkit for real-time web application monitoring, logging,
and access control. With GitLab's offering, the [OWASP's Core Rule Set](https://www.modsecurity.org/CRS/Documentation/),
which provides generic attack detection capabilities, is automatically applied.
This feature:
...
...
@@ -275,6 +285,12 @@ This feature:
To enable ModSecurity, check the **Enable Web Application Firewall** checkbox
when installing your [Ingress application](#ingress).
If this is your first time using GitLab's WAF, we recommend you follow the
@@ -139,10 +139,10 @@ The following table depicts the various user permission levels in a project.
| Force push to protected branches (*4*) | | | | | |
| Remove protected branches (*4*) | | | | | |
\* Owner permission is only available at the group or personal namespace level (and for instance admins) and is inherited by its projects.
(*1*): Guest users are able to perform this action on public and internal projects, but not private projects.
(*2*): Guest users can only view the confidential issues they created themselves.
(*3*): If **Public pipelines** is enabled in **Project Settings > CI/CD**.
\* Owner permission is only available at the group or personal namespace level (and for instance admins) and is inherited by its projects.
(*1*): Guest users are able to perform this action on public and internal projects, but not private projects.
(*2*): Guest users can only view the confidential issues they created themselves.
(*3*): If **Public pipelines** is enabled in **Project Settings > CI/CD**.
(*4*): Not allowed for Guest, Reporter, Developer, Maintainer, or Owner. See [Protected Branches](./project/protected_branches.md).
(*5*): If the [branch is protected](./project/protected_branches.md#using-the-allowed-to-merge-and-allowed-to-push-settings), this depends on the access Developers and Maintainers are given.
...
...
@@ -166,10 +166,10 @@ Maintainers and Developers from pushing to a protected branch. Read through the
[Allowed to Merge and Allowed to Push settings](project/protected_branches.md#using-the-allowed-to-merge-and-allowed-to-push-settings)
to learn more.
### Cycle Analytics permissions
### Value Stream Analytics permissions
Find the current permissions on the Cycle Analytics dashboard on
the [documentation on Cycle Analytics permissions](analytics/cycle_analytics.md#permissions).
Find the current permissions on the Value Stream Analytics dashboard, as described in
@@ -267,11 +267,11 @@ msgstr "Eventos de notificaciones personalizadas"
msgid "Custom notification levels are the same as participating levels. With custom notification levels you will also receive notifications for select events. To find out more, check out %{notification_link}."
msgstr "Los niveles de notificación personalizados son los mismos que los niveles participantes. Con los niveles de notificación personalizados, también recibirá notificaciones para eventos seleccionados. Para obtener más información, consulte %{notification_link}."
msgid "Cycle Analytics"
msgstr "Cycle Analytics"
msgid "Value Stream Analytics"
msgstr "Value Stream Analytics"
msgid "Cycle Analytics gives an overview of how much time it takes to go from idea to production in your project."
msgstr "Cycle Analytics ofrece una visión general de cuánto tiempo tarda en pasar de idea a producción en su proyecto."
msgid "Value Stream Analytics gives an overview of how much time it takes to go from idea to production in your project."
msgstr "Value Stream Analytics ofrece una visión general de cuánto tiempo tarda en pasar de idea a producción en su proyecto."
