Skip to content

G
gitlab-ce
Commit e972f813 authored by Evan Read's avatar Evan Read
Browse files
Options

Merge branch 'docs/refactor-include-ci-yaml' into 'master' 

Refactor `include` CI yaml docs

Closes #56235

See merge request gitlab-org/gitlab-ce!24355
parents 14c13c80 6c8aa675
Hide whitespace changes
Inline Side-by-side
Showing with 287 additions and 248 deletions
doc/ci/yaml/README.md
View file @ e972f813
...@@ -78,118 +78,6 @@ A job is defined by a list of parameters that define the job behavior. ...@@ -78,118 +78,6 @@ A job is defined by a list of parameters that define the job behavior.
| [retry](#retry) | no | Define when and how many times a job can be auto-retried in case of a failure | | [retry](#retry) | no | Define when and how many times a job can be auto-retried in case of a failure |
| [parallel](#parallel) | no | Defines how many instances of a job should be run in parallel | | [parallel](#parallel) | no | Defines how many instances of a job should be run in parallel |
### `extends`
> Introduced in GitLab 11.3.
`extends` defines an entry name that a job that uses `extends` is going to
inherit from.
It is an alternative to using [YAML anchors](#anchors) and is a little
more flexible and readable:
```yaml
.tests:
script: rake test
stage: test
only:
refs:
- branches
rspec:
extends: .tests
script: rake rspec
only:
variables:
- $RSPEC
```
In the example above, the `rspec` job inherits from the `.tests` template job.
GitLab will perform a reverse deep merge based on the keys. GitLab will:
- Merge the `rspec` contents into `.tests` recursively.
- Not merge the values of the keys.
This results in the following `rspec` job:
```yaml
rspec:
script: rake rspec
stage: test
only:
refs:
- branches
variables:
- $RSPEC
```
NOTE: **Note:**
Note that `script: rake test` has been overwritten by `script: rake rspec`.
If you do want to include the `rake test`, have a look at [before_script-and-after_script](#before_script-and-after_script).
`.tests` in this example is a [hidden key](#hidden-keys-jobs), but it's
possible to inherit from regular jobs as well.
`extends` supports multi-level inheritance, however it is not recommended to
use more than three levels. The maximum nesting level that is supported is 10.
The following example has two levels of inheritance:
```yaml
.tests:
only:
- pushes
.rspec:
extends: .tests
script: rake rspec
rspec 1:
variables:
RSPEC_SUITE: '1'
extends: .rspec
rspec 2:
variables:
RSPEC_SUITE: '2'
extends: .rspec
spinach:
extends: .tests
script: rake spinach
```
`extends` works across configuration files combined with [`include`](#include).
### `pages`
`pages` is a special job that is used to upload static content to GitLab that
can be used to serve your website. It has a special syntax, so the two
requirements below must be met:
1. Any static content must be placed under a `public/` directory
1. `artifacts` with a path to the `public/` directory must be defined
The example below simply moves all files from the root of the project to the
`public/` directory. The `.public` workaround is so `cp` doesn't also copy
`public/` to itself in an infinite loop:
```yaml
pages:
stage: deploy
script:
- mkdir .public
- cp -r * .public
- mv .public public
artifacts:
paths:
- public
only:
- master
```
Read more on [GitLab Pages user documentation](../../user/project/pages/index.md).
## `image` and `services` ## `image` and `services`
This allows to specify a custom Docker image and a list of services that can be This allows to specify a custom Docker image and a list of services that can be
...@@ -1645,193 +1533,214 @@ test: ...@@ -1645,193 +1533,214 @@ test:
## `include` ## `include`
> Introduced in [GitLab Premium](https://about.gitlab.com/pricing/) 10.5. > - Introduced in [GitLab Premium](https://about.gitlab.com/pricing/) 10.5.
> Available for Starter, Premium and Ultimate since 10.6. > - Available for Starter, Premium and Ultimate since 10.6.
> Behaviour expanded in GitLab 10.8 to allow more flexible overriding. > - [Moved](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/21603)
> [Moved](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/21603) to GitLab Core in 11.4.
to GitLab Core in 11.4
> In GitLab 11.7, support for [including GitLab-supplied templates directly](https://gitlab.com/gitlab-org/gitlab-ce/issues/53445) and support for [including templates from another repository](https://gitlab.com/gitlab-org/gitlab-ce/issues/53903) was added.
Using the `include` keyword, you can allow the inclusion of external YAML files. Using the `include` keyword, you can allow the inclusion of external YAML files.
`include` requires the external YAML file to have the extensions `.yml` or `.yaml`,
otherwise the external file will not be included.
In the following example, the content of `.before-script-template.yml` will be The files defined in `include` are:
automatically fetched and evaluated along with the content of `.gitlab-ci.yml`:
```yaml - Deep merged with those in `.gitlab-ci.yml`.
# Content of https://gitlab.com/awesome-project/raw/master/.before-script-template.yml - Always evaluated first and merged with the content of `.gitlab-ci.yml`,
regardless of the position of the `include` keyword.
before_script: TIP: **Tip:**
- apt-get update -qq && apt-get install -y -qq sqlite3 libsqlite3-dev nodejs Use merging to customize and override included CI/CD configurations with local
- gem install bundler --no-document definitions.
- bundle install --jobs $(nproc) "${FLAGS[@]}"
```
```yaml Recursive includes are not supported. Your external files should not use the
# Content of .gitlab-ci.yml `include` keyword as it will be ignored.
include: 'https://gitlab.com/awesome-project/raw/master/.before-script-template.yml' NOTE: **Note:**
Using YAML aliases across different YAML files sourced by `include` is not
supported. You must only refer to aliases in the same file. Instead
of using YAML anchors, you can use the [`extends` keyword](#extends).
rspec: `include` supports four include methods:
script:
- bundle exec rspec
```
NOTE: **Note:** - [`local`](#includelocal)
`include` requires the external YAML files to have the extensions `.yml` or `.yaml`. - [`file`](#includefile)
The external file will not be included if the extension is missing. - [`template`](#includetemplate)
- [`remote`](#includeremote)
You can include your extra YAML file either as a single string or See [usage examples](#include-examples).
as an array of multiple values. You can also use full paths or
relative URLs. The following examples are both valid:
```yaml ### `include:local`
# Single string
include: '/templates/.after-script-template.yml' `include:local` includes a file from the same repository as `.gitlab-ci.yml`.
``` It's referenced using full paths relative to the root directory (`/`).
```yaml You can only use files that are currently tracked by Git on the same branch
# Single string your configuration file is on. In other words, when using a `include:local`, make
sure that both `.gitlab-ci.yml` and the local file are on the same branch.
NOTE: **Note:**
Including local files through Git submodules paths is not supported.
Example:
```yaml
include: include:
file: '/templates/.after-script-template.yml' - local: '/templates/.gitlab-ci-template.yml'
``` ```
```yaml ### `include:file`
# Array
> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/53903) in GitLab 11.7.
To include files from another private project under the same GitLab instance,
use `include:file`. This file is referenced using full paths relative to the
root directory (`/`). For example:
```yaml
include: include:
- 'https://gitlab.com/awesome-project/raw/master/.before-script-template.yml' - project: 'my-group/my-project'
- '/templates/.after-script-template.yml' file: '/templates/.gitlab-ci-template.yml'
``` ```
```yaml You can also specify `ref`, with the default being the `HEAD` of the project:
# Array mixed syntax
```yaml
include: include:
- 'https://gitlab.com/awesome-project/raw/master/.before-script-template.yml' - project: 'my-group/my-project'
- '/templates/.after-script-template.yml' ref: master
- template: Auto-DevOps.gitlab-ci.yml file: '/templates/.gitlab-ci-template.yml'
- project: 'my-group/my-project'
ref: v1.0.0
file: '/templates/.gitlab-ci-template.yml'
- project: 'my-group/my-project'
ref: 787123b47f14b552955ca2786bc9542ae66fee5b # Git SHA
file: '/templates/.gitlab-ci-template.yml'
``` ```
```yaml ### `include:template`
# Array
> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/53445) in GitLab 11.7.
`include:template` can be used to include `.gitlab-ci.yml` templates that are
[shipped with GitLab](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/gitlab/ci/templates).
For example:
```yaml
# File sourced from GitLab's template collection
include: include:
- remote: 'https://gitlab.com/awesome-project/raw/master/.before-script-template.yml'
- local: '/templates/.after-script-template.yml'
- template: Auto-DevOps.gitlab-ci.yml - template: Auto-DevOps.gitlab-ci.yml
``` ```
--- ### `include:remote`
`include` supports four types of files: `include:remote` can be used to include a file from a different location,
using HTTP/HTTPS, referenced by using the full URL. The remote file must be
publicly accessible through a simple GET request as authentication schemas
in the remote URL is not supported. For example:
- **local** to the same repository, referenced by using full paths in the same ```yaml
repository, with `/` being the root directory. For example: include:
- remote: 'https://gitlab.com/awesome-project/raw/master/.gitlab-ci-template.yml'
```
```yaml NOTE: **Note for GitLab admins:**
# Within the repository In order to include files from another repository inside your local network,
include: '/templates/.gitlab-ci-template.yml' you may need to enable the **Allow requests to the local network from hooks and services** checkbox
``` located in the **Admin area > Settings > Network > Outbound requests** section.
Or using: ### `include` examples
```yaml Here are a few more `include` examples.
# Within the repository
include:
local: '/templates/.gitlab-ci-template.yml'
```
NOTE: **Note:** #### Single string or array of multiple values
You can only use files that are currently tracked by Git on the same branch
your configuration file is. In other words, when using a **local file**, make
sure that both `.gitlab-ci.yml` and the local file are on the same branch.
NOTE: **Note:** You can include your extra YAML file(s) either as a single string or
We don't support the inclusion of local files through Git submodules paths. an array of multiple values. The following examples are all valid.
- **file** from another repository, referenced by using full paths in the same Single string with the `include:local` method implied:
repository, with `/` being the root directory. For example:
```yaml ```yaml
include: include: '/templates/.after-script-template.yml'
project: 'my-group/my-project' ```
file: '/templates/.gitlab-ci-template.yml'
```
You can also specify `ref:`. The default `ref:` is the `HEAD` of the project: Array with `include` method implied:
```yaml ```yaml
include: include:
- project: 'my-group/my-project' - 'https://gitlab.com/awesome-project/raw/master/.before-script-template.yml'
ref: master - '/templates/.after-script-template.yml'
file: '/templates/.gitlab-ci-template.yml' ```
- project: 'my-group/my-project' Single string with `include` method specified explicitly:
ref: v1.0.0
file: '/templates/.gitlab-ci-template.yml'
- project: 'my-group/my-project' ```yaml
ref: 787123b47f14b552955ca2786bc9542ae66fee5b # git sha include:
file: '/templates/.gitlab-ci-template.yml' remote: 'https://gitlab.com/awesome-project/raw/master/.before-script-template.yml'
``` ```
- **remote** in a different location, accessed using HTTP/HTTPS, referenced Array with `include:remote` being the single item:
using the full URL. For example:
```yaml ```yaml
# File sourced from outside repository include:
include: 'https://gitlab.com/awesome-project/raw/master/.gitlab-ci-template.yml' - remote: 'https://gitlab.com/awesome-project/raw/master/.before-script-template.yml'
``` ```
Or using: Array with multiple `include` methods specified explicitly:
```yaml ```yaml
# File sourced from outside repository include:
include: - remote: 'https://gitlab.com/awesome-project/raw/master/.before-script-template.yml'
remote: 'https://gitlab.com/awesome-project/raw/master/.gitlab-ci-template.yml' - local: '/templates/.after-script-template.yml'
``` - template: Auto-DevOps.gitlab-ci.yml
```
NOTE: **Note:** Array mixed syntax:
The remote file must be publicly accessible through a simple GET request, as we don't support authentication schemas in the remote URL.
NOTE: **Note:** ```yaml
In order to include files from another repository inside your local network, include:
you may need to enable the **Allow requests to the local network from hooks and services** checkbox - 'https://gitlab.com/awesome-project/raw/master/.before-script-template.yml'
located in the **Settings > Network > Outbound requests** section within the **Admin area**. - '/templates/.after-script-template.yml'
- template: Auto-DevOps.gitlab-ci.yml
- project: 'my-group/my-project'
ref: master
file: '/templates/.gitlab-ci-template.yml'
```
- **template** included with GitLab. For example: #### Re-using a `before_script` template
```yaml In the following example, the content of `.before-script-template.yml` will be
# File sourced from GitLab's template collection automatically fetched and evaluated along with the content of `.gitlab-ci.yml`.
include:
template: Auto-DevOps.gitlab-ci.yml
```
NOTE: **Note:** Content of `https://gitlab.com/awesome-project/raw/master/.before-script-template.yml`:
Templates included this way are sourced from [lib/gitlab/ci/templates](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/gitlab/ci/templates).
--- ```yaml
before_script:
- apt-get update -qq && apt-get install -y -qq sqlite3 libsqlite3-dev nodejs
- gem install bundler --no-document
- bundle install --jobs $(nproc) "${FLAGS[@]}"
```
Content of `.gitlab-ci.yml`:
Since GitLab 10.8 we are now deep merging the files defined in `include` ```yaml
with those in `.gitlab-ci.yml`. Files defined by `include` are always include: 'https://gitlab.com/awesome-project/raw/master/.before-script-template.yml'
evaluated first and merged with the content of `.gitlab-ci.yml`, no
matter the position of the `include` keyword. You can take advantage of
merging to customize and override details in included CI
configurations with local definitions.
NOTE: **Note:** rspec:
The recursive includes are not supported, meaning your external files script:
should not use the `include` keyword, as it will be ignored. - bundle exec rspec
```
#### Overriding external template values
The following example shows specific YAML-defined variables and details of the The following example shows specific YAML-defined variables and details of the
`production` job from an include file being customized in `.gitlab-ci.yml`. `production` job from an include file being customized in `.gitlab-ci.yml`.
```yaml Content of `https://company.com/autodevops-template.yml`:
# Content of https://company.com/autodevops-template.yml
```yaml
variables: variables:
POSTGRES_USER: user POSTGRES_USER: user
POSTGRES_PASSWORD: testing_password POSTGRES_PASSWORD: testing_password
...@@ -1849,9 +1758,9 @@ production: ...@@ -1849,9 +1758,9 @@ production:
- master - master
``` ```
```yaml Content of `.gitlab-ci.yml`:
# Content of .gitlab-ci.yml
```yaml
include: 'https://company.com/autodevops-template.yml' include: 'https://company.com/autodevops-template.yml'
image: alpine:latest image: alpine:latest
...@@ -1878,11 +1787,11 @@ with the environment url of the `production` job defined in ...@@ -1878,11 +1787,11 @@ with the environment url of the `production` job defined in
The merging lets you extend and override dictionary mappings, but The merging lets you extend and override dictionary mappings, but
you cannot add or modify items to an included array. For example, to add you cannot add or modify items to an included array. For example, to add
an additional item to the production job script, you must repeat the an additional item to the production job script, you must repeat the
existing script items. existing script items:
```yaml Content of `https://company.com/autodevops-template.yml`:
# Content of https://company.com/autodevops-template.yml
```yaml
production: production:
stage: production stage: production
script: script:
...@@ -1890,9 +1799,9 @@ production: ...@@ -1890,9 +1799,9 @@ production:
- deploy - deploy
``` ```
```yaml Content of `.gitlab-ci.yml`:
# Content of .gitlab-ci.yml
```yaml
include: 'https://company.com/autodevops-template.yml' include: 'https://company.com/autodevops-template.yml'
stages: stages:
...@@ -1909,10 +1818,140 @@ In this case, if `install_dependencies` and `deploy` were not repeated in ...@@ -1909,10 +1818,140 @@ In this case, if `install_dependencies` and `deploy` were not repeated in
`.gitlab-ci.yml`, they would not be part of the script for the `production` `.gitlab-ci.yml`, they would not be part of the script for the `production`
job in the combined CI configuration. job in the combined CI configuration.
## `extends`
> Introduced in GitLab 11.3.
`extends` defines an entry name that a job that uses `extends` is going to
inherit from.
It is an alternative to using [YAML anchors](#anchors) and is a little
more flexible and readable:
```yaml
.tests:
script: rake test
stage: test
only:
refs:
- branches
rspec:
extends: .tests
script: rake rspec
only:
variables:
- $RSPEC
```
In the example above, the `rspec` job inherits from the `.tests` template job.
GitLab will perform a reverse deep merge based on the keys. GitLab will:
- Merge the `rspec` contents into `.tests` recursively.
- Not merge the values of the keys.
This results in the following `rspec` job:
```yaml
rspec:
script: rake rspec
stage: test
only:
refs:
- branches
variables:
- $RSPEC
```
NOTE: **Note:** NOTE: **Note:**
We currently do not support using YAML aliases across different YAML files Note that `script: rake test` has been overwritten by `script: rake rspec`.
sourced by `include`. You must only refer to aliases in the same file. Instead
of using YAML anchors you can use [`extends` keyword](#extends). If you do want to include the `rake test`, see [before_script-and-after_script](#before_script-and-after_script).
`.tests` in this example is a [hidden key](#hidden-keys-jobs), but it's
possible to inherit from regular jobs as well.
`extends` supports multi-level inheritance, however it is not recommended to
use more than three levels. The maximum nesting level that is supported is 10.
The following example has two levels of inheritance:
```yaml
.tests:
only:
- pushes
.rspec:
extends: .tests
script: rake rspec
rspec 1:
variables:
RSPEC_SUITE: '1'
extends: .rspec
rspec 2:
variables:
RSPEC_SUITE: '2'
extends: .rspec
spinach:
extends: .tests
script: rake spinach
```
## Using `extends` and `include` together
`extends` works across configuration files combined with `include`.
For example, if you have a local `included.yml` file:
```yaml
.template:
script:
- echo Hello!
```
Then, in `.gitlab-ci.yml` you can use it like this:
```yaml
include: included.yml
useTemplate:
image: alpine
extends: .template
```
This will run a job called `useTemplate` that runs `echo Hello!` as defined in
the `.template` job, and uses the `alpine` Docker image as defined in the local job.
## `pages`
`pages` is a special job that is used to upload static content to GitLab that
can be used to serve your website. It has a special syntax, so the two
requirements below must be met:
- Any static content must be placed under a `public/` directory.
- `artifacts` with a path to the `public/` directory must be defined.
The example below simply moves all files from the root of the project to the
`public/` directory. The `.public` workaround is so `cp` doesn't also copy
`public/` to itself in an infinite loop:
```yaml
pages:
stage: deploy
script:
- mkdir .public
- cp -r * .public
- mv .public public
artifacts:
paths:
- public
only:
- master
```
Read more on [GitLab Pages user documentation](../../user/project/pages/index.md).
## `variables` ## `variables`
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7