Commit 6c0e8406 authored by Marcel Amirault's avatar Marcel Amirault

Merge branch 'selhorn-resource-group' into 'master'

Put resource_group into new format

See merge request gitlab-org/gitlab!73582
parents e17dbefd cb5cec9b
...@@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w ...@@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
description: Control the job concurrency in GitLab CI/CD description: Control the job concurrency in GitLab CI/CD
--- ---
# Resource Group **(FREE)** # Resource group **(FREE)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15536) in GitLab 12.7. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15536) in GitLab 12.7.
...@@ -13,7 +13,7 @@ By default, pipelines in GitLab CI/CD run in parallel. The parallelization is an ...@@ -13,7 +13,7 @@ By default, pipelines in GitLab CI/CD run in parallel. The parallelization is an
the feedback loop in merge requests, however, there are some situations that the feedback loop in merge requests, however, there are some situations that
you may want to limit the concurrency on deployment you may want to limit the concurrency on deployment
jobs to run them one by one. jobs to run them one by one.
Resource Group allows you to strategically control Use resource groups to strategically control
the concurrency of the jobs for optimizing your continuous deployments workflow with safety. the concurrency of the jobs for optimizing your continuous deployments workflow with safety.
## Add a resource group ## Add a resource group
...@@ -139,9 +139,59 @@ Depending on the process mode of the resource group: ...@@ -139,9 +139,59 @@ Depending on the process mode of the resource group:
- `deploy-1`, `deploy-2`, and `deploy-3` do not run in parallel. - `deploy-1`, `deploy-2`, and `deploy-3` do not run in parallel.
- `deploy-3` runs first, `deploy-2` runs second and `deploy-1` runs last. - `deploy-3` runs first, `deploy-2` runs second and `deploy-1` runs last.
## Pipeline-level concurrency control with Cross-Project/Parent-Child pipelines ## Pipeline-level concurrency control with cross-project/parent-child pipelines
See the how to [control the pipeline concurrency in cross-project pipelines](../yaml/index.md#pipeline-level-concurrency-control-with-cross-projectparent-child-pipelines). > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/39057) in GitLab 13.9.
You can define `resource_group` for downstream pipelines that are sensitive to concurrent
executions. The [`trigger` keyword](../yaml/index.md#trigger) can trigger downstream pipelines and the
[`resource_group` keyword](../yaml/index.md#resource_group) can co-exist with it. `resource_group` is useful to control the
concurrency of deployment pipelines, while other jobs can continue to run concurrently.
The following example has two pipeline configurations in a project. When a pipeline starts running,
non-sensitive jobs are executed first and aren't affected by concurrent executions in other
pipelines. However, GitLab ensures that there are no other deployment pipelines running before
triggering a deployment (child) pipeline. If other deployment pipelines are running, GitLab waits
until those pipelines finish before running another one.
```yaml
# .gitlab-ci.yml (parent pipeline)
build:
stage: build
script: echo "Building..."
test:
stage: test
script: echo "Testing..."
deploy:
stage: deploy
trigger:
include: deploy.gitlab-ci.yml
strategy: depend
resource_group: AWS-production
```
```yaml
# deploy.gitlab-ci.yml (child pipeline)
stages:
- provision
- deploy
provision:
stage: provision
script: echo "Provisioning..."
deployment:
stage: deploy
script: echo "Deploying..."
```
You must define [`strategy: depend`](../yaml/index.md#triggerstrategy)
with the `trigger` keyword. This ensures that the lock isn't released until the downstream pipeline
finishes.
## API ## API
......
...@@ -3941,92 +3941,37 @@ In this example, a new pipeline causes a running pipeline to be: ...@@ -3941,92 +3941,37 @@ In this example, a new pipeline causes a running pipeline to be:
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15536) in GitLab 12.7. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15536) in GitLab 12.7.
Sometimes running multiple jobs at the same time in an environment Use `resource_group` to create a [resource group](../resource_groups/index.md) that
can lead to errors during the deployment. ensures a job is mutually exclusive across different pipelines for the same project.
To avoid these errors, use the `resource_group` attribute to make sure that For example, if multiple jobs that belong to the same resource group are queued simultaneously,
the runner doesn't run certain jobs concurrently. Resource groups behave similar only one of the jobs starts. The other jobs wait until the `resource_group` is free.
to semaphores in other programming languages.
When the `resource_group` keyword is defined for a job in the `.gitlab-ci.yml` file, Resource groups behave similar to semaphores in other programming languages.
job executions are mutually exclusive across different pipelines for the same project.
If multiple jobs belonging to the same resource group are enqueued simultaneously,
only one of the jobs is picked by the runner. The other jobs wait until the
`resource_group` is free.
For example:
```yaml
deploy-to-production:
script: deploy
resource_group: production
```
In this case, two `deploy-to-production` jobs in two separate pipelines can never run at the same time. As a result,
you can ensure that concurrent deployments never happen to the production environment.
You can define multiple resource groups per environment. For example, You can define multiple resource groups per environment. For example,
when deploying to physical devices, you may have multiple physical devices. Each device when deploying to physical devices, you might have multiple physical devices. Each device
can be deployed to, but there can be only one deployment per device at any given time. can be deployed to, but only one deployment can occur per device at any given time.
The `resource_group` value can only contain letters, digits, `-`, `_`, `/`, `$`, `{`, `}`, `.`, and spaces. **Keyword type**: Job keyword. You can use it only as part of a job.
It can't start or end with `/`.
For more information, see [Resource Group documentation](../resource_groups/index.md).
#### Pipeline-level concurrency control with Cross-Project/Parent-Child pipelines
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/39057) in GitLab 13.9.
You can define `resource_group` for downstream pipelines that are sensitive to concurrent **Possible inputs**: Only letters, digits, `-`, `_`, `/`, `$`, `{`, `}`, `.`, and spaces.
executions. The [`trigger` keyword](#trigger) can trigger downstream pipelines. The It can't start or end with `/`.
[`resource_group` keyword](#resource_group) can co-exist with it. This is useful to control the
concurrency for deployment pipelines, while running non-sensitive jobs concurrently.
The following example has two pipeline configurations in a project. When a pipeline starts running, **Example of `resource_group`**:
non-sensitive jobs are executed first and aren't affected by concurrent executions in other
pipelines. However, GitLab ensures that there are no other deployment pipelines running before
triggering a deployment (child) pipeline. If other deployment pipelines are running, GitLab waits
until those pipelines finish before running another one.
```yaml ```yaml
# .gitlab-ci.yml (parent pipeline) deploy-to-production:
script: deploy
build: resource_group: production
stage: build
script: echo "Building..."
test:
stage: test
script: echo "Testing..."
deploy:
stage: deploy
trigger:
include: deploy.gitlab-ci.yml
strategy: depend
resource_group: AWS-production
``` ```
```yaml In this example, two `deploy-to-production` jobs in two separate pipelines can never run at the same time. As a result,
# deploy.gitlab-ci.yml (child pipeline) you can ensure that concurrent deployments never happen to the production environment.
stages:
- provision
- deploy
provision:
stage: provision
script: echo "Provisioning..."
deployment: **Related topics**:
stage: deploy
script: echo "Deploying..."
```
You must define [`strategy: depend`](#triggerstrategy) - [Pipeline-level concurrency control with cross-project/parent-child pipelines](../resource_groups/index.md#pipeline-level-concurrency-control-with-cross-projectparent-child-pipelines).
with the `trigger` keyword. This ensures that the lock isn't released until the downstream pipeline
finishes.
### `release` ### `release`
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment