Commit 0af4c86c authored by Marcel Amirault's avatar Marcel Amirault Committed by Suzanne Selhorn

Create a new page about controlling CI/CD jobs

parent 8d03fe1b
......@@ -400,8 +400,8 @@ stop_review:
when: manual
```
Both jobs must have the same [`rules`](../yaml/README.md#onlyexcept-basic)
or [`only/except`](../yaml/README.md#onlyexcept-basic) configuration. Otherwise,
Both jobs must have the same [`rules`](../yaml/README.md#only--except)
or [`only/except`](../yaml/README.md#only--except) configuration. Otherwise,
the `stop_review` job might not be included in all pipelines that include the
`deploy_review` job, and you cannot trigger `action: stop` to stop the environment automatically.
......
---
stage: Verify
group: Continuous Integration
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# Choose when to run jobs **(FREE)**
When a new pipeline starts, GitLab checks the pipeline configuration to determine
which jobs should run in that pipeline. You can configure jobs to run depending on
the status of variables, the pipeline type, and so on.
To configure a job to be included or excluded from certain pipelines, you can use:
- [`rules`](../yaml/README.md#rules)
- [`only`](../yaml/README.md#only--except)
- [`except`](../yaml/README.md#only--except)
Use [`needs`](../yaml/README.md#needs) to configure a job to run as soon as the
earlier jobs it depends on finish running.
## Specify when jobs run with `only` and `except`
You can use [`only`](../yaml/README.md#only--except) and [`except`](../yaml/README.md#only--except)
to control when to add jobs to pipelines.
- Use `only` to define when a job runs.
- Use `except` to define when a job **does not** run.
### `only:refs` / `except:refs` examples
`only` or `except` used without `refs` is the same as
[`only:refs` / `except/refs`](../yaml/README.md#onlyrefs--exceptrefs)
In the following example, `job` runs only for:
- Git tags
- [Triggers](../triggers/README.md#trigger-token)
- [Scheduled pipelines](../pipelines/schedules.md)
```yaml
job:
# use special keywords
only:
- tags
- triggers
- schedules
```
To execute jobs only for the parent repository and not forks:
```yaml
job:
only:
- branches@gitlab-org/gitlab
except:
- main@gitlab-org/gitlab
- /^release/.*$/@gitlab-org/gitlab
```
This example runs `job` for all branches on `gitlab-org/gitlab`,
except `main` and branches that start with `release/`.
### `only: variables` / `except: variables` examples
You can use `except:variables` to exclude jobs based on a commit message:
```yaml
end-to-end:
script: rake test:end-to-end
except:
variables:
- $CI_COMMIT_MESSAGE =~ /skip-end-to-end-tests/
```
You can use [parentheses](../variables/README.md#parentheses) with `&&` and `||`
to build more complicated variable expressions:
```yaml
job1:
script:
- echo This rule uses parentheses.
only:
variables:
- ($CI_COMMIT_BRANCH == "master" || $CI_COMMIT_BRANCH == "develop") && $MY_VARIABLE
```
### `only:changes` / `except:changes` examples
You can skip a job if a change is detected in any file with a
`.md` extension in the root directory of the repository:
```yaml
build:
script: npm run build
except:
changes:
- "*.md"
```
If you change multiple files, but only one file ends in `.md`,
the `build` job is still skipped. The job does not run for any of the files.
Read more about how to use `only:changes` and `except:changes`:
- [New branches or tags *without* pipelines for merge requests](#use-onlychanges-without-pipelines-for-merge-requests).
- [Scheduled pipelines](#use-onlychanges-with-scheduled-pipelines).
#### Use `only:changes` with pipelines for merge requests
With [pipelines for merge requests](../merge_request_pipelines/index.md),
it's possible to define a job to be created based on files modified
in a merge request.
Use this keyword with `only: [merge_requests]` so GitLab can find the correct base
SHA of the source branch. File differences are correctly calculated from any further
commits, and all changes in the merge requests are properly tested in pipelines.
For example:
```yaml
docker build service one:
script: docker build -t my-service-one-image:$CI_COMMIT_REF_SLUG .
only:
refs:
- merge_requests
changes:
- Dockerfile
- service-one/**/*
```
In this scenario, if a merge request changes
files in the `service-one` directory or the `Dockerfile`, GitLab creates
the `docker build service one` job.
For example:
```yaml
docker build service one:
script: docker build -t my-service-one-image:$CI_COMMIT_REF_SLUG .
only:
changes:
- Dockerfile
- service-one/**/*
```
In this example, the pipeline might fail because of changes to a file in `service-one/**/*`.
A later commit that doesn't have changes in `service-one/**/*`
but does have changes to the `Dockerfile` can pass. The job
only tests the changes to the `Dockerfile`.
GitLab checks the **most recent pipeline** that **passed**. If the merge request is mergeable,
it doesn't matter that an earlier pipeline failed because of a change that has not been corrected.
When you use this configuration, ensure that the most recent pipeline
properly corrects any failures from previous pipelines.
#### Use `only:changes` without pipelines for merge requests
Without [pipelines for merge requests](../merge_request_pipelines/index.md), pipelines
run on branches or tags that don't have an explicit association with a merge request.
In this case, a previous SHA is used to calculate the diff, which is equivalent to `git diff HEAD~`.
This can result in some unexpected behavior, including:
- When pushing a new branch or a new tag to GitLab, the policy always evaluates to true.
- When pushing a new commit, the changed files are calculated by using the previous commit
as the base SHA.
#### Use `only:changes` with scheduled pipelines
`only:changes` always evaluates as true in [Scheduled pipelines](../pipelines/schedules.md).
All files are considered to have changed when a scheduled pipeline runs.
### Combine multiple keywords with `only` or `except`
If you use multiple keywords with `only` or `except`, the keywords are evaluated
as a single conjoined expression. That is:
- `only:` includes the job if **all** of the keys have at least one condition that matches.
- `except:` excludes the job if **any** of the keys have at least one condition that matches.
With `only`, individual keys are logically joined by an `AND`. A job is added to
the pipeline if the following is true:
- `(any listed refs are true) AND (any listed variables are true) AND (any listed changes are true) AND (any chosen Kubernetes status matches)`
In the following example, the `test` job is only created when **all** of the following are true:
- The pipeline is [scheduled](../pipelines/schedules.md) **or** runs for `main`.
- The `variables` keyword matches.
- The `kubernetes` service is active on the project.
```yaml
test:
script: npm run test
only:
refs:
- main
- schedules
variables:
- $CI_COMMIT_MESSAGE =~ /run-end-to-end-tests/
kubernetes: active
```
With `except`, individual keys are logically joined by an `OR`. A job is **not**
added if the following is true:
- `(any listed refs are true) OR (any listed variables are true) OR (any listed changes are true) OR (a chosen Kubernetes status matches)`
In the following example, the `test` job is **not** created when **any** of the following are true:
- The pipeline runs for the `main` branch.
- There are changes to the `README.md` file in the root directory of the repository.
```yaml
test:
script: npm run test
except:
refs:
- main
changes:
- "README.md"
```
## Regular expressions
The `@` symbol denotes the beginning of a ref's repository path.
To match a ref name that contains the `@` character in a regular expression,
you must use the hex character code match `\x40`.
Only the tag or branch name can be matched by a regular expression.
The repository path, if given, is always matched literally.
To match the tag or branch name,
the entire ref name part of the pattern must be a regular expression surrounded by `/`.
For example, you can't use `issue-/.*/` to match all tag names or branch names
that begin with `issue-`, but you can use `/issue-.*/`.
Regular expression flags must be appended after the closing `/`. Pattern matching
is case-sensitive by default. Use the `i` flag modifier, like `/pattern/i`, to make
a pattern case-insensitive:
```yaml
job:
# use regexp
only:
- /^issue-.*$/i
# use special keyword
except:
- branches
```
Use anchors `^` and `$` to avoid the regular expression
matching only a substring of the tag name or branch name.
For example, `/^issue-.*$/` is equivalent to `/^issue-/`,
while just `/issue/` would also match a branch called `severe-issues`.
### `only` / `except` regex syntax
In GitLab 11.9.4, GitLab began internally converting the regexp used
in `only` and `except` keywords to [RE2](https://github.com/google/re2/wiki/Syntax).
[RE2](https://github.com/google/re2/wiki/Syntax) limits the set of available features
due to computational complexity, and some features, like negative lookaheads, became unavailable.
Only a subset of features provided by [Ruby Regexp](https://ruby-doc.org/core/Regexp.html)
are now supported.
From GitLab 11.9.7 to GitLab 12.0, GitLab provided a feature flag to
let you use unsafe regexp syntax. After migrating to safe syntax, you should disable
this feature flag again:
```ruby
Feature.enable(:allow_unsafe_ruby_regexp)
```
......@@ -137,7 +137,7 @@ save resources.
#### Excluding certain branches
Pipelines for merge requests require special treatment when
using [`only`/`except`](../yaml/README.md#onlyexcept-basic). Unlike ordinary
using [`only`/`except`](../yaml/README.md#only--except). Unlike ordinary
branch refs (for example `refs/heads/my-feature-branch`), merge request refs
use a special Git reference that looks like `refs/merge-requests/:iid/head`. Because
of this, the following configuration will **not** work as expected:
......@@ -153,7 +153,7 @@ Instead, you can use the
[`$CI_COMMIT_REF_NAME` predefined environment
variable](../variables/predefined_variables.md) in
combination with
[`only:variables`](../yaml/README.md#onlyvariablesexceptvariables) to
[`only:variables`](../yaml/README.md#onlyvariables--exceptvariables) to
accomplish this behavior:
```yaml
......
......@@ -349,8 +349,8 @@ variable entry.
GitLab does support a [`when` keyword](../yaml/README.md#when) which is used to indicate when a job should be
run in case of (or despite) failure, but most of the logic for controlling pipelines can be found in
our very powerful [`only/except` rules system](../yaml/README.md#onlyexcept-basic)
(see also our [advanced syntax](../yaml/README.md#onlyexcept-basic)):
our very powerful [`only/except` rules system](../yaml/README.md#only--except)
(see also our [advanced syntax](../yaml/README.md#only--except)):
```yaml
my_job:
......
......@@ -67,7 +67,7 @@ When using:
- CI/CD Variables or [`rules`](yaml/README.md#rulesif) to control job behavior, the value of
the [`$CI_PIPELINE_SOURCE` predefined variable](variables/predefined_variables.md) is
`pipeline` for multi-project pipeline triggered through the API with `CI_JOB_TOKEN`.
- [`only/except`](yaml/README.md#onlyexcept-basic) to control job behavior, use the
- [`only/except`](yaml/README.md#only--except) to control job behavior, use the
`pipelines` keyword.
## Creating multi-project pipelines from `.gitlab-ci.yml`
......@@ -113,7 +113,7 @@ When using:
the [`$CI_PIPELINE_SOURCE` predefined variable](variables/predefined_variables.md) is
`pipeline` for multi-project pipelines triggered with a bridge job (using the
[`trigger:`](yaml/README.md#trigger) keyword).
- [`only/except`](yaml/README.md#onlyexcept-basic) to control job behavior, use the
- [`only/except`](yaml/README.md#only--except) to control job behavior, use the
`pipelines` keyword.
In the example, `staging` is marked as successful as soon as a downstream pipeline
......
......@@ -38,7 +38,7 @@ set of concurrently running child pipelines, but within the same project:
Child pipelines work well with other GitLab CI/CD features:
- Use [`only: changes`](yaml/README.md#onlychangesexceptchanges) to trigger pipelines only when
- Use [`only: changes`](yaml/README.md#onlychanges--exceptchanges) to trigger pipelines only when
certain files change. This is useful for monorepos, for example.
- Since the parent pipeline in `.gitlab-ci.yml` and the child pipeline run as normal
pipelines, they can have their own behaviors and sequencing in relation to triggers.
......
......@@ -65,7 +65,7 @@ GitLab CI/CD so that they can be used in your [`.gitlab-ci.yml` file](../../ci/y
To configure a job to be executed only when the pipeline has been
scheduled (or the opposite), use
[only and except](../yaml/README.md#onlyexcept-basic) configuration keywords.
[only and except](../yaml/README.md#only--except) configuration keywords.
In the example below `make world` runs in scheduled pipelines, and `make build` runs in pipelines that are not scheduled:
......
......@@ -26,7 +26,7 @@ depending on which trigger method is used.
| `pipeline` | Using the `trigger:` keyword in the CI/CD configuration file, or using the trigger API with `$CI_JOB_TOKEN`. |
| `trigger` | Using the trigger API using a generated trigger token |
This also applies when using the `pipelines` or `triggers` keywords with the legacy [`only/except` basic syntax](../yaml/README.md#onlyexcept-basic).
This also applies when using the `pipelines` or `triggers` keywords with the legacy [`only/except` basic syntax](../yaml/README.md#only--except).
### Trigger token
......
......@@ -123,7 +123,7 @@ This is usually caused by the `rules` configuration, and there are several ways
#### A job is not in the pipeline
GitLab determines if a job is added to a pipeline based on the [`only/except`](yaml/README.md#onlyexcept-basic)
GitLab determines if a job is added to a pipeline based on the [`only/except`](yaml/README.md#only--except)
or [`rules`](yaml/README.md#rules) defined for the job. If it didn't run, it's probably
not evaluating as you expect.
......@@ -150,7 +150,7 @@ A common reason a job is added to a pipeline unexpectedly is because the `change
keyword always evaluates to true in certain cases. For example, `changes` is always
true in certain pipeline types, including scheduled pipelines and pipelines for tags.
The `changes` keyword is used in combination with [`only/except`](yaml/README.md#onlychangesexceptchanges)
The `changes` keyword is used in combination with [`only/except`](yaml/README.md#onlychanges--exceptchanges)
or [`rules`](yaml/README.md#ruleschanges)). It's recommended to use `changes` with
`rules` or `only/except` configuration that ensures the job is only added to branch
pipelines or merge request pipelines.
......
......@@ -642,7 +642,7 @@ CI/CD variables with multi-line values are not supported.
## CI/CD variable expressions
> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/37397) in GitLab 10.7 for [the `only` and `except` CI keywords](../yaml/README.md#onlyexcept-advanced)
> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/37397) in GitLab 10.7 for [the `only` and `except` CI keywords](../yaml/README.md#onlyvariables--exceptvariables)
> - [Expanded](https://gitlab.com/gitlab-org/gitlab/-/issues/27863) in GitLab 12.3 with [the `rules` keyword](../yaml/README.md#rules)
Use variable expressions to limit which jobs are created
......@@ -651,7 +651,7 @@ in a pipeline after changes are pushed to GitLab.
In `.gitlab-ci.yml`, variable expressions work with both:
- [`rules`](../yaml/README.md#rules), which is the recommended approach, and
- [`only` and `except`](../yaml/README.md#onlyexcept-basic), which are candidates for deprecation.
- [`only` and `except`](../yaml/README.md#only--except), which are candidates for deprecation.
This is particularly useful in combination with variables and triggered
pipeline variables.
......@@ -672,7 +672,7 @@ If any of the conditions in `variables` evaluates to true when using `only`,
a new job is created. If any of the expressions evaluates to true
when `except` is being used, a job is not created.
This follows the usual rules for [`only` / `except` policies](../yaml/README.md#onlyexcept-advanced).
This follows the usual rules for [`only` / `except` policies](../yaml/README.md#onlyvariables--exceptvariables).
### Syntax of CI/CD variable expressions
......
......@@ -36,14 +36,14 @@ The keywords available for jobs are:
| [`coverage`](#coverage) | Code coverage settings for a given job. |
| [`dependencies`](#dependencies) | Restrict which artifacts are passed to a specific job by providing a list of jobs to fetch artifacts from. |
| [`environment`](#environment) | Name of an environment to which the job deploys. |
| [`except`](#onlyexcept-basic) | Limit when jobs are not created. |
| [`except`](#only--except) | Control when jobs are not created. |
| [`extends`](#extends) | Configuration entries that this job inherits from. |
| [`image`](#image) | Use Docker images. |
| [`include`](#include) | Include external YAML files. |
| [`inherit`](#inherit) | Select which global defaults all jobs inherit. |
| [`interruptible`](#interruptible) | Defines if a job can be canceled when made redundant by a newer run. |
| [`needs`](#needs) | Execute jobs earlier than the stage ordering. |
| [`only`](#onlyexcept-basic) | Limit when jobs are created. |
| [`only`](#only--except) | Control when jobs are created. |
| [`pages`](#pages) | Upload the result of a job to use with GitLab Pages. |
| [`parallel`](#parallel) | How many instances of a job should be run in parallel. |
| [`release`](#release) | Instructs the runner to generate a [release](../../user/project/releases/index.md) object. |
......@@ -1104,7 +1104,7 @@ is either included or excluded from the pipeline, depending on the configuration
The job can also have [certain attributes](#rules-attributes)
added to it.
`rules` replaces [`only/except`](#onlyexcept-basic) and they can't be used together
`rules` replaces [`only/except`](#only--except) and they can't be used together
in the same job. If you configure one job to use both keywords, the linter returns a
`key may not be used with rules` error.
......@@ -1137,8 +1137,8 @@ Available rule clauses are:
| Clause | Description |
|----------------------------|------------------------------------------------------------------------------------------------------------------------------------|
| [`if`](#rulesif) | Add or exclude jobs from a pipeline by evaluating an `if` statement. Similar to [`only:variables`](#onlyvariablesexceptvariables). |
| [`changes`](#ruleschanges) | Add or exclude jobs from a pipeline based on what files are changed. Same as [`only:changes`](#onlychangesexceptchanges). |
| [`if`](#rulesif) | Add or exclude jobs from a pipeline by evaluating an `if` statement. Similar to [`only:variables`](#onlyvariables--exceptvariables). |
| [`changes`](#ruleschanges) | Add or exclude jobs from a pipeline based on what files are changed. Same as [`only:changes`](#onlychanges--exceptchanges). |
| [`exists`](#rulesexists) | Add or exclude jobs from a pipeline based on the presence of specific files. |
Rules are evaluated in order until a match is found. If a match is found, the attributes
......@@ -1295,7 +1295,7 @@ job-with-rules:
For every change pushed to the branch, duplicate pipelines run. One
branch pipeline runs a single job (`job-with-no-rules`), and one merge request pipeline
runs the other job (`job-with-rules`). Jobs with no rules default
to [`except: merge_requests`](#onlyexcept-basic), so `job-with-no-rules`
to [`except: merge_requests`](#only--except), so `job-with-no-rules`
runs in all cases except merge requests.
#### `rules:if`
......@@ -1344,7 +1344,7 @@ Some details regarding the logic that determines the `when` for the job:
##### Common `if` clauses for `rules`
For behavior similar to the [`only`/`except` keywords](#onlyexcept-basic), you can
For behavior similar to the [`only`/`except` keywords](#only--except), you can
check the value of the `$CI_PIPELINE_SOURCE` variable:
| Value | Description |
......@@ -1406,7 +1406,7 @@ Other commonly used variables for `if` clauses:
Use `rules:changes` to specify when to add a job to a pipeline by checking for
changes to specific files.
`rules: changes` works the same way as [`only: changes` and `except: changes`](#onlychangesexceptchanges).
`rules: changes` works the same way as [`only: changes` and `except: changes`](#onlychanges--exceptchanges).
It accepts an array of paths. You should use `rules: changes` only with branch
pipelines or merge request pipelines. For example, it's common to use `rules: changes`
with merge request pipelines:
......@@ -1437,7 +1437,7 @@ rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH
```
To implement a rule similar to [`except:changes`](#onlychangesexceptchanges),
To implement a rule similar to [`except:changes`](#onlychanges--exceptchanges),
use `when: never`.
WARNING:
......@@ -1597,318 +1597,162 @@ WARNING:
[Before GitLab 13.3](https://gitlab.com/gitlab-org/gitlab/-/issues/230938),
rules that use both `||` and `&&` may evaluate with an unexpected order of operations.
### `only`/`except` (basic)
### `only` / `except`
NOTE:
`only` and `except` are not being actively developed. To define when
to add jobs to pipelines, use [`rules`](#rules).
`only` and `except` are not being actively developed. [`rules`](#rules) is the preferred
keyword to control when to add jobs to pipelines.
`only` and `except` are two keywords that determine when to add jobs to pipelines:
You can use `only` and `except` to control when to add jobs to pipelines.
1. `only` defines the names of branches and tags the job runs for.
1. `except` defines the names of branches and tags the job does
**not** run for.
- Use `only` to define when a job runs.
- Use `except` to define when a job **does not** run.
A few rules apply to the usage of job policy:
Four keywords can be used with `only` and `except`:
- `only` and `except` are inclusive. If both `only` and `except` are defined
in a job specification, the ref is filtered by `only` and `except`.
- `only` and `except` can use regular expressions ([supported regexp syntax](#supported-onlyexcept-regexp-syntax)).
- `only` and `except` can specify a repository path to filter jobs for forks.
- [`refs`](#onlyrefs--exceptrefs)
- [`variables`](#onlyvariables--exceptvariables)
- [`changes`](#onlychanges--exceptchanges)
- [`kubernetes`](#onlykubernetes--exceptkubernetes)
In addition, `only` and `except` can use these keywords:
See [control jobs with `only` and `except`](../jobs/job_control.md#specify-when-jobs-run-with-only-and-except)
for more details and examples.
| **Value** | **Description** |
|--------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `api` | For pipelines triggered by the [pipelines API](../../api/pipelines.md#create-a-new-pipeline). |
| `branches` | When the Git reference for a pipeline is a branch. |
| `chat` | For pipelines created by using a [GitLab ChatOps](../chatops/index.md) command. |
| `external` | When you use CI services other than GitLab. |
| `external_pull_requests` | When an external pull request on GitHub is created or updated (See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests)). |
| `merge_requests` | For pipelines created when a merge request is created or updated. Enables [merge request pipelines](../merge_request_pipelines/index.md), [merged results pipelines](../merge_request_pipelines/pipelines_for_merged_results/index.md), and [merge trains](../merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md). |
| `pipelines` | For [multi-project pipelines](../multi_project_pipelines.md) created by [using the API with `CI_JOB_TOKEN`](../multi_project_pipelines.md#triggering-multi-project-pipelines-through-api), or the [`trigger`](#trigger) keyword. |
| `pushes` | For pipelines triggered by a `git push` event, including for branches and tags. |
| `schedules` | For [scheduled pipelines](../pipelines/schedules.md). |
| `tags` | When the Git reference for a pipeline is a tag. |
| `triggers` | For pipelines created by using a [trigger token](../triggers/README.md#trigger-token). |
| `web` | For pipelines created by using **Run pipeline** button in the GitLab UI, from the project's **CI/CD > Pipelines** section. |
#### `only:refs` / `except:refs`
Scheduled pipelines run on specific branches, so jobs configured with `only: branches`
run on scheduled pipelines too. Add `except: schedules` to prevent jobs with `only: branches`
from running on scheduled pipelines.
Use the `only:refs` and `except:refs` keywords to control when to add jobs to a
pipeline based on branch names or pipeline types.
In the following example, `job` runs only for refs that start with `issue-`.
All branches are skipped:
**Keyword type**: Job keyword. You can use it only as part of a job.
```yaml
job:
# use regexp
only:
- /^issue-.*$/
# use special keyword
except:
- branches
```
Pattern matching is case-sensitive by default. Use the `i` flag modifier, like
`/pattern/i`, to make a pattern case-insensitive:
**Possible inputs**: An array including any number of:
```yaml
job:
# use regexp
only:
- /^issue-.*$/i
# use special keyword
except:
- branches
```
- Branch names, for example `main` or `my-feature-branch`.
- [Regular expressions](../jobs/job_control.md#only--except-regex-syntax)
that match against branch names, for example `/^feature-.*/`.
- The following keywords:
In the following example, `job` runs only for:
| **Value** | **Description** |
| -------------------------|-----------------|
| `api` | For pipelines triggered by the [pipelines API](../../api/pipelines.md#create-a-new-pipeline). |
| `branches` | When the Git reference for a pipeline is a branch. |
| `chat` | For pipelines created by using a [GitLab ChatOps](../chatops/index.md) command. |
| `external` | When you use CI services other than GitLab. |
| `external_pull_requests` | When an external pull request on GitHub is created or updated (See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests)). |
| `merge_requests` | For pipelines created when a merge request is created or updated. Enables [merge request pipelines](../merge_request_pipelines/index.md), [merged results pipelines](../merge_request_pipelines/pipelines_for_merged_results/index.md), and [merge trains](../merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md). |
| `pipelines` | For [multi-project pipelines](../multi_project_pipelines.md) created by [using the API with `CI_JOB_TOKEN`](../multi_project_pipelines.md#triggering-multi-project-pipelines-through-api), or the [`trigger`](#trigger) keyword. |
| `pushes` | For pipelines triggered by a `git push` event, including for branches and tags. |
| `schedules` | For [scheduled pipelines](../pipelines/schedules.md). |
| `tags` | When the Git reference for a pipeline is a tag. |
| `triggers` | For pipelines created by using a [trigger token](../triggers/README.md#trigger-token). |
| `web` | For pipelines created by using **Run pipeline** button in the GitLab UI, from the project's **CI/CD > Pipelines** section. |
- Git tags
- [Triggers](../triggers/README.md#trigger-token)
- [Scheduled pipelines](../pipelines/schedules.md)
```yaml
job:
# use special keywords
only:
- tags
- triggers
- schedules
```
To execute jobs only for the parent repository and not forks:
```yaml
job:
only:
- branches@gitlab-org/gitlab
except:
- master@gitlab-org/gitlab
- /^release/.*$/@gitlab-org/gitlab
```
This example runs `job` for all branches on `gitlab-org/gitlab`,
except `master` and branches that start with `release/`.
If a job does not have an `only` rule, `only: ['branches', 'tags']` is set by
default. If the job does not have an `except` rule, it's empty.
For example, `job1` and `job2` are essentially the same:
**Example of `only:refs` and `except:refs`**:
```yaml
job1:
script: echo 'test'
job2:
script: echo 'test'
only: ['branches', 'tags']
```
#### Regular expressions
The `@` symbol denotes the beginning of a ref's repository path.
To match a ref name that contains the `@` character in a regular expression,
you must use the hex character code match `\x40`.
Only the tag or branch name can be matched by a regular expression.
The repository path, if given, is always matched literally.
To match the tag or branch name,
the entire ref name part of the pattern must be a regular expression surrounded by `/`.
For example, you can't use `issue-/.*/` to match all tag names or branch names
that begin with `issue-`, but you can use `/issue-.*/`.
Regular expression flags must be appended after the closing `/`.
NOTE:
Use anchors `^` and `$` to avoid the regular expression
matching only a substring of the tag name or branch name.
For example, `/^issue-.*$/` is equivalent to `/^issue-/`,
while just `/issue/` would also match a branch called `severe-issues`.
#### Supported `only`/`except` regexp syntax
In GitLab 11.9.4, GitLab began internally converting the regexp used
in `only` and `except` keywords to [RE2](https://github.com/google/re2/wiki/Syntax).
[RE2](https://github.com/google/re2/wiki/Syntax) limits the set of available features
due to computational complexity, and some features, like negative lookaheads, became unavailable.
Only a subset of features provided by [Ruby Regexp](https://ruby-doc.org/core/Regexp.html)
are now supported.
From GitLab 11.9.7 to GitLab 12.0, GitLab provided a feature flag to
let you use unsafe regexp syntax. After migrating to safe syntax, you should disable
this feature flag again:
```ruby
Feature.enable(:allow_unsafe_ruby_regexp)
```
### `only`/`except` (advanced)
GitLab supports multiple strategies, and it's possible to use an
array or a hash configuration scheme.
Four keys are available:
- `refs`
- `variables`
- `changes`
- `kubernetes`
If you use multiple keys under `only` or `except`, the keys are evaluated as a
single conjoined expression. That is:
- `only:` includes the job if **all** of the keys have at least one condition that matches.
- `except:` excludes the job if **any** of the keys have at least one condition that matches.
With `only`, individual keys are logically joined by an `AND`. A job is added to
the pipeline if the following is true:
- `(any listed refs are true) AND (any listed variables are true) AND (any listed changes are true) AND (any chosen Kubernetes status matches)`
In the following example, the `test` job is `only` created when **all** of the following are true:
- The pipeline is [scheduled](../pipelines/schedules.md) **or** runs for `master`.
- The `variables` keyword matches.
- The `kubernetes` service is active on the project.
```yaml
test:
script: npm run test
script: echo
only:
refs:
- master
- schedules
variables:
- $CI_COMMIT_MESSAGE =~ /run-end-to-end-tests/
kubernetes: active
```
With `except`, individual keys are logically joined by an `OR`. A job is **not**
added if the following is true:
- `(any listed refs are true) OR (any listed variables are true) OR (any listed changes are true) OR (a chosen Kubernetes status matches)`
In the following example, the `test` job is **not** created when **any** of the following are true:
- The pipeline runs for the `master` branch.
- There are changes to the `README.md` file in the root directory of the repository.
- main
- /^issue-.*$/
- merge_requests
```yaml
test:
script: npm run test
job2:
script: echo
except:
refs:
- master
changes:
- "README.md"
- main
- /^stable-branch.*$/
- schedules
```
#### `only:refs`/`except:refs`
> `refs` policy introduced in GitLab 10.0.
The `refs` strategy can take the same values as the
[simplified only/except configuration](#onlyexcept-basic).
**Additional details:**
In the following example, the `deploy` job is created only when the
pipeline is [scheduled](../pipelines/schedules.md) or runs for the `master` branch:
- Scheduled pipelines run on specific branches, so jobs configured with `only: branches`
run on scheduled pipelines too. Add `except: schedules` to prevent jobs with `only: branches`
from running on scheduled pipelines.
- `only` or `except` used without any other keywords are equivalent to `only: refs`
or `except: refs`. For example, the following two jobs configurations have the same
behavior:
```yaml
deploy:
only:
refs:
- master
- schedules
```
```yaml
job1:
script: echo
only:
- branches
#### `only:kubernetes`/`except:kubernetes`
job2:
script: echo
only:
refs:
- branches
```
> `kubernetes` policy introduced in GitLab 10.0.
- If a job does not use `only`, `except`, or [`rules`](#rules), then `only` is set to `branches`
and `tags` by default.
The `kubernetes` strategy accepts only the `active` keyword.
For example, `job1` and `job2` are equivalent:
In the following example, the `deploy` job is created only when the
Kubernetes service is active in the project:
```yaml
job1:
script: echo 'test'
```yaml
deploy:
only:
kubernetes: active
```
job2:
script: echo 'test'
only:
- branches
- tags
```
#### `only:variables`/`except:variables`
#### `only:variables` / `except:variables`
> `variables` policy introduced in GitLab 10.7.
Use the `only:variables` or `except:variables` keywords to control when to add jobs
to a pipeline, based on the status of [CI/CD variables](../variables/README.md).
The `variables` keyword defines variable expressions.
**Keyword type**: Job keyword. You can use it only as part of a job.
These expressions determine whether or not a job should be created.
**Possible inputs**: An array of [CI/CD variable expressions](../variables/README.md#cicd-variable-expressions).
Examples of variable expressions:
**Example of `only:variables`**:
```yaml
deploy:
script: cap staging deploy
only:
refs:
- branches
variables:
- $RELEASE == "staging"
- $STAGING
```
Another use case is excluding jobs depending on a commit message:
**Related topics**:
```yaml
end-to-end:
script: rake test:end-to-end
except:
variables:
- $CI_COMMIT_MESSAGE =~ /skip-end-to-end-tests/
```
- [`only:variables` and `except:variables` examples](../jobs/job_control.md#only-variables--except-variables-examples).
You can use [parentheses](../variables/README.md#parentheses) with `&&` and `||` to build more complicated variable expressions.
[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/230938) in GitLab 13.3:
#### `only:changes` / `except:changes`
```yaml
job1:
script:
- echo This rule uses parentheses.
only:
variables:
- ($CI_COMMIT_BRANCH == "master" || $CI_COMMIT_BRANCH == "develop") && $MY_VARIABLE
```
#### `only:changes`/`except:changes`
> `changes` policy [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/19232) in GitLab 11.4.
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/19232) in GitLab 11.4.
Use the `changes` keyword with `only` to run a job, or with `except` to skip a job,
when a Git push event modifies a file.
Use `only:changes` with pipelines triggered by the following refs only:
Use `changes` in pipelines with the following refs:
- `branches`
- `external_pull_requests`
- `merge_requests` (see additional details about [using `only:changes` with pipelines for merge requests](#use-onlychanges-with-pipelines-for-merge-requests))
- `merge_requests` (see additional details about [using `only:changes` with pipelines for merge requests](../jobs/job_control.md#use-onlychanges-with-pipelines-for-merge-requests))
WARNING:
In pipelines with [sources other than the three above](../variables/predefined_variables.md)
`changes` can't determine if a given file is new or old and always returns `true`.
You can configure jobs to use `only: changes` with other `only: refs` keywords. However,
those jobs ignore the changes and always run.
**Keyword type**: Job keyword. You can use it only as part of a job.
**Possible inputs**: An array including any number of:
In the following example, when you push commits to an existing branch, the `docker build` job
runs only if any of these files change:
- Paths to files.
- Wildcard paths for single directories, for example `path/to/directory/*`, or a directory
and all its subdirectories, for example `path/to/directory/**/*`.
- Wildcard ([glob](https://en.wikipedia.org/wiki/Glob_(programming))) paths for all
files with the same extension or multiple extensions, for example `*.md` or `path/to/directory/*.{rb,py,sh}`.
- Wildcard paths to files in the root directory, or all directories, wrapped in double quotes.
For example `"*.json"` or `"**/*.json"`.
- The `Dockerfile` file.
- Files in the `docker/scripts/` directory.
- Files and subdirectories in the `dockerfiles` directory.
- Files with `rb`, `py`, `sh` extensions in the `more_scripts` directory.
Example of `only:changes`:
```yaml
docker build:
......@@ -1923,110 +1767,40 @@ docker build:
- more_scripts/*.{rb,py,sh}
```
WARNING:
If you use `only:changes` with [only allow merge requests to be merged if the pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds),
you should [also use `only:merge_requests`](#use-onlychanges-with-pipelines-for-merge-requests). Otherwise it may not work as expected.
You can also use [glob](https://en.wikipedia.org/wiki/Glob_(programming))
patterns to match multiple files in either the root directory
of the repository, or in _any_ directory in the repository. However, they must be wrapped
in double quotes or GitLab can't parse them:
```yaml
test:
script: npm run test
only:
refs:
- branches
changes:
- "*.json"
- "**/*.sql"
```
**Additional details**:
You can skip a job if a change is detected in any file with a
`.md` extension in the root directory of the repository:
- If you use refs other than `branches`, `external_pull_requests`, or `merge_requests`,
`changes` can't determine if a given file is new or old and always returns `true`.
- If you use `only: changes` with other refs, jobs ignore the changes and always run.
- If you use `except: changes` with other refs, jobs ignore the changes and never run.
```yaml
build:
script: npm run build
except:
changes:
- "*.md"
```
**Related topics**:
If you change multiple files, but only one file ends in `.md`,
the `build` job is still skipped. The job does not run for any of the files.
- [`only: changes` and `except: changes` examples](../jobs/job_control.md#onlychanges--exceptchanges-examples).
- If you use `changes` with [only allow merge requests to be merged if the pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds),
you should [also use `only:merge_requests`](../jobs/job_control.md#use-onlychanges-with-pipelines-for-merge-requests).
- Use `changes` with [new branches or tags *without* pipelines for merge requests](../jobs/job_control.md#use-onlychanges-without-pipelines-for-merge-requests).
- Use `changes` with [scheduled pipelines](../jobs/job_control.md#use-onlychanges-with-scheduled-pipelines).
Read more about how to use this feature with:
#### `only:kubernetes` / `except:kubernetes`
- [New branches or tags *without* pipelines for merge requests](#use-onlychanges-without-pipelines-for-merge-requests).
- [Scheduled pipelines](#use-onlychanges-with-scheduled-pipelines).
Use `only:kubernetes` or `except:kubernetes` to control if jobs are added to the pipeline
when the Kubernetes service is active in the project.
##### Use `only:changes` with pipelines for merge requests
**Keyword type**: Job-specific. You can use it only as part of a job.
With [pipelines for merge requests](../merge_request_pipelines/index.md),
it's possible to define a job to be created based on files modified
in a merge request.
**Possible inputs**: The `kubernetes` strategy accepts only the `active` keyword.
Use this keyword with `only: [merge_requests]` so GitLab can find the correct base
SHA of the source branch. File differences are correctly calculated from any further
commits, and all changes in the merge requests are properly tested in pipelines.
For example:
Example of `only:kubernetes`:
```yaml
docker build service one:
script: docker build -t my-service-one-image:$CI_COMMIT_REF_SLUG .
only:
refs:
- merge_requests
changes:
- Dockerfile
- service-one/**/*
```
In this scenario, if a merge request changes
files in the `service-one` directory or the `Dockerfile`, GitLab creates
the `docker build service one` job.
For example:
```yaml
docker build service one:
script: docker build -t my-service-one-image:$CI_COMMIT_REF_SLUG .
deploy:
only:
changes:
- Dockerfile
- service-one/**/*
kubernetes: active
```
In this example, the pipeline might fail because of changes to a file in `service-one/**/*`.
A later commit that doesn't have changes in `service-one/**/*`
but does have changes to the `Dockerfile` can pass. The job
only tests the changes to the `Dockerfile`.
GitLab checks the **most recent pipeline** that **passed**. If the merge request is mergeable,
it doesn't matter that an earlier pipeline failed because of a change that has not been corrected.
When you use this configuration, ensure that the most recent pipeline
properly corrects any failures from previous pipelines.
##### Use `only:changes` without pipelines for merge requests
Without [pipelines for merge requests](../merge_request_pipelines/index.md), pipelines
run on branches or tags that don't have an explicit association with a merge request.
In this case, a previous SHA is used to calculate the diff, which is equivalent to `git diff HEAD~`.
This can result in some unexpected behavior, including:
- When pushing a new branch or a new tag to GitLab, the policy always evaluates to true.
- When pushing a new commit, the changed files are calculated by using the previous commit
as the base SHA.
##### Use `only:changes` with scheduled pipelines
`only:changes` always evaluates as true in [Scheduled pipelines](../pipelines/schedules.md).
All files are considered to have changed when a scheduled pipeline runs.
In this example, the `deploy` job runs only when the Kubernetes service is active
in the project.
### `needs`
......@@ -2295,7 +2069,7 @@ This feature might not be available to you. Check the **version history** note a
To need a job that sometimes does not exist in the pipeline, add `optional: true`
to the `needs` configuration. If not defined, `optional: false` is the default.
Jobs that use [`rules`](#rules), [`only`, or `except`](#onlyexcept-basic), might
Jobs that use [`rules`](#rules), [`only`, or `except`](#only--except), might
not always exist in a pipeline. When the pipeline starts, it checks the `needs`
relationships before running. Without `optional: true`, needs relationships that
point to a job that does not exist stops the pipeline from starting and causes a pipeline
......@@ -2769,8 +2543,8 @@ The `stop_review_app` job is **required** to have the following keywords defined
- `environment:name`
- `environment:action`
Additionally, both jobs should have matching [`rules`](#onlyexcept-basic)
or [`only/except`](#onlyexcept-basic) configuration.
Additionally, both jobs should have matching [`rules`](#only--except)
or [`only/except`](#only--except) configuration.
In the examples above, if the configuration is not identical:
......
......@@ -127,7 +127,7 @@ job2:
#### Use `rules` instead of `only` or `except`
Avoid using [`only` or `except`](../../ci/yaml/README.md#onlyexcept-basic) if possible.
Avoid using [`only` or `except`](../../ci/yaml/README.md#only--except) if possible.
Only and except is not being developed any more, and [`rules`](../../ci/yaml/README.md#rules)
is now the preferred syntax:
......
......@@ -244,8 +244,8 @@ include:
See the [Auto DevOps template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml) for information on available jobs.
WARNING:
Auto DevOps templates using the [`only`](../../ci/yaml/README.md#onlyexcept-basic) or
[`except`](../../ci/yaml/README.md#onlyexcept-basic) syntax have switched
Auto DevOps templates using the [`only`](../../ci/yaml/README.md#only--except) or
[`except`](../../ci/yaml/README.md#only--except) syntax have switched
to the [`rules`](../../ci/yaml/README.md#rules) syntax, starting in
[GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/213336).
If your `.gitlab-ci.yml` extends these Auto DevOps templates and override the `only` or
......
......@@ -270,7 +270,7 @@ container_scanning_new:
```
WARNING:
GitLab 13.0 and later doesn't support [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic).
GitLab 13.0 and later doesn't support [`only` and `except`](../../../ci/yaml/README.md#only--except).
When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules)
instead.
......
......@@ -656,7 +656,7 @@ To view details of vulnerabilities detected by DAST:
### Customizing the DAST settings
WARNING:
Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic)
Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#only--except)
is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead.
The DAST settings can be changed through CI/CD variables by using the
......
......@@ -130,7 +130,7 @@ configuration, the last mention of the variable takes precedence.
### Overriding dependency scanning jobs
WARNING:
Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic)
Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#only--except)
is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead.
To override a job definition (for example, to change properties like `variables` or `dependencies`),
......
......@@ -451,7 +451,7 @@ Found errors in your .gitlab-ci.yml:
```
This error appears when the included job's `rules` configuration has been [overridden](sast/index.md#overriding-sast-jobs)
with [the deprecated `only` or `except` syntax.](../../ci/yaml/README.md#onlyexcept-basic)
with [the deprecated `only` or `except` syntax.](../../ci/yaml/README.md#only--except)
To fix this issue, you must either:
- [Transition your `only/except` syntax to `rules`](#transitioning-your-onlyexcept-syntax-to-rules).
......@@ -462,7 +462,7 @@ To fix this issue, you must either:
#### Transitioning your `only/except` syntax to `rules`
When overriding the template to control job execution, previous instances of
[`only` or `except`](../../ci/yaml/README.md#onlyexcept-basic) are no longer compatible
[`only` or `except`](../../ci/yaml/README.md#only--except) are no longer compatible
and must be transitioned to [the `rules` syntax](../../ci/yaml/README.md#rules).
If your override is aimed at limiting jobs to only run on `master`, the previous syntax
......
......@@ -222,7 +222,7 @@ the pipeline configuration, the last mention of the variable takes precedence.
### Overriding SAST jobs
WARNING:
Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic)
Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#only--except)
is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead.
To override a job definition, (for example, change properties like `variables` or `dependencies`),
......
......@@ -169,7 +169,7 @@ declare a job with the same name as the SAST job to override. Place this new job
inclusion and specify any additional keys under it.
WARNING:
Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic)
Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#only--except)
is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead.
#### GIT_DEPTH
......
......@@ -180,7 +180,7 @@ directory of your project.
### Overriding the template
WARNING:
Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic)
Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#only--except)
is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead.
If you want to override the job definition (for example, change properties like
......
......@@ -67,7 +67,7 @@ You should be careful to configure CI/CD so that pipelines run for every merge r
### Limitations
When this setting is enabled, a merge request is prevented from being merged if there
is no pipeline. This may conflict with some use cases where [`only/except`](../../../ci/yaml/README.md#onlyexcept-advanced)
is no pipeline. This may conflict with some use cases where [`only/except`](../../../ci/yaml/README.md#only--except)
or [`rules`](../../../ci/yaml/README.md#rules) are used and they don't generate any pipelines.
You should ensure that [there is always a pipeline](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/54226)
......
......@@ -129,7 +129,7 @@ See this document for a [step-by-step guide](getting_started/pages_from_scratch.
Remember that GitLab Pages are by default branch/tag agnostic and their
deployment relies solely on what you specify in `.gitlab-ci.yml`. You can limit
the `pages` job with the [`only` parameter](../../../ci/yaml/README.md#onlyexcept-basic),
the `pages` job with the [`only` parameter](../../../ci/yaml/README.md#only--except),
whenever a new commit is pushed to a branch used specifically for your
pages.
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment