1.[Trigger variables][triggers] or [scheduled pipeline variables](../../user/project/pipelines/schedules.md#making-use-of-scheduled-pipeline-variables)(take precedence over all)
1. Project-level [secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables)
1. Group-level [secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables)
This document describes the usage of `.gitlab-ci.yml`, the file that is used by
GitLab Runner to manage your project's jobs.
If you want a quick introduction to GitLab CI, follow our
[quick start guide](../quick_start/README.md).
## .gitlab-ci.yml
From version 7.12, GitLab CI uses a [YAML](https://en.wikipedia.org/wiki/YAML)
file (`.gitlab-ci.yml`) for the project configuration. It is placed in the root
of your repository and contains definitions of how your project should be built.
If you want a quick introduction to GitLab CI, follow our
[quick start guide](../quick_start/README.md).
## Jobs
The YAML file defines a set of jobs with constraints stating when they should
be run. The jobs are defined as top-level elements with a name and always have
to contain at least the `script` clause:
be run. You can specify an unlimited number of jobs which are defined as
top-level elements with an arbitrary name and always have to contain at least
the `script` clause.
```yaml
job1:
...
...
@@ -24,9 +25,8 @@ job2:
script:"execute-script-for-job2"
```
The above example is the simplest possible CI configuration with two separate
The above example is the simplest possible CI/CD configuration with two separate
jobs, where each of the jobs executes a different command.
Of course a command can execute code directly (`./configure;make;make install`)
or run a script (`test.sh`) in the repository.
...
...
@@ -34,78 +34,115 @@ Jobs are picked up by [Runners](../runners/README.md) and executed within the
environment of the Runner. What is important, is that each job is run
independently from each other.
The YAML syntax allows for using more complex job specifications than in the
above example:
Each job must have a unique name, but there are a few **reserved `keywords` that
cannot be used as job names**:
```yaml
image:ruby:2.1
services:
-postgres
-`image`
-`services`
-`stages`
-`types`
-`before_script`
-`after_script`
-`variables`
-`cache`
before_script:
-bundle install
A job is defined by a list of parameters that define the job behavior.
after_script:
-rm secrets
| Keyword | Required | Description |
|---------------|----------|-------------|
| script | yes | Defines a shell script which is executed by Runner |
| image | no | Use docker image, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) |
| services | no | Use docker services, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) |
| stage | no | Defines a job stage (default: `test`) |
| type | no | Alias for `stage` |
| variables | no | Define job variables on a job level |
| only | no | Defines a list of git refs for which job is created |
| except | no | Defines a list of git refs for which job is not created |
| tags | no | Defines a list of tags which are used to select Runner |
| allow_failure | no | Allow job to fail. Failed job doesn't contribute to commit status |
| when | no | Define when to run job. Can be `on_success`, `on_failure`, `always` or `manual` |
| dependencies | no | Define other jobs that a job depends on so that you can pass artifacts between them|
| artifacts | no | Define list of [job artifacts](#artifacts) |
| cache | no | Define list of files that should be cached between subsequent runs |
| before_script | no | Override a set of commands that are executed before job |
| after_script | no | Override a set of commands that are executed after job |
| environment | no | Defines a name of environment to which deployment is done by this job |
| coverage | no | Define code coverage settings for a given job |
| retry | no | Define how many times a job can be auto-retried in case of a failure |
stages:
-build
-test
-deploy
### `pages`
job1:
stage:build
`pages` is a special job that is used to upload static content to GitLab that
can be used to serve your website. It has a special syntax, so the two
requirements below must be met:
1. Any static content must be placed under a `public/` directory
1.`artifacts` with a path to the `public/` directory must be defined
The example below simply moves all files from the root of the project to the
`public/` directory. The `.public` workaround is so `cp` doesn't also copy
`public/` to itself in an infinite loop:
```
pages:
stage: deploy
script:
-execute-script-for-job1
- mkdir .public
- cp -r * .public
- mv .public public
artifacts:
paths:
- public
only:
- master
tags:
-docker
```
There are a few reserved `keywords` that **cannot** be used as job names:
| Keyword | Required | Description |
|---------------|----------|-------------|
| image | no | Use docker image, covered in [Use Docker](../docker/README.md) |
| services | no | Use docker services, covered in [Use Docker](../docker/README.md) |
| stages | no | Define build stages |
| types | no | Alias for `stages` (deprecated) |
| before_script | no | Define commands that run before each job's script |
| after_script | no | Define commands that run after each job's script |
| variables | no | Define build variables |
| cache | no | Define list of files that should be cached between subsequent runs |
Read more on [GitLab Pages user documentation](../../user/project/pages/index.md).
### image and services
## `image` and `services`
This allows to specify a custom Docker image and a list of services that can be
used for time of the job. The configuration of this feature is covered in
[a separate document](../docker/README.md).
### before_script
`before_script` is used to define the command that should be run before all
jobs, including deploy jobs, but after the restoration of artifacts. This can
be an array or a multi-line string.
### after_script
## `before_script` and `after_script`
> Introduced in GitLab 8.7 and requires Gitlab Runner v1.2
`before_script` is used to define the command that should be run before all
jobs, including deploy jobs, but after the restoration of [artifacts](#artifacts).
This can be an array or a multi-line string.
`after_script` is used to define the command that will be run after for all
jobs, including failed ones. This has to be an array or a multi-line string.
> **Note:**
The `before_script` and the main `script` are concatenated and run in a single context/container.
The `after_script` is run separately, so depending on the executor, changes done
outside of the working tree might not be visible, e.g. software installed in the
`before_script`.
### stages
It's possible to overwrite the globally defined `before_script` and `after_script`
if you set it per-job:
`stages` is used to define stages that can be used by jobs.
The specification of `stages` allows for having flexible multi stage pipelines.
```yaml
before_script:
-global before script
job:
before_script:
-execute this instead of global before script
script:
-my command
after_script:
-execute this after my script
```
## `stages`
`stages` is used to define stages that can be used by jobs and is defined
globally.
The specification of `stages` allows for having flexible multi stage pipelines.
The ordering of elements in `stages` defines the ordering of jobs' execution:
1. Jobs of the same stage are run in parallel.
...
...
@@ -126,288 +163,53 @@ stages:
1. If all jobs of `test` succeed, the `deploy` jobs are executed in parallel.
1. If all jobs of `deploy` succeed, the commit is marked as `passed`.
1. If any of the previous jobs fails, the commit is marked as `failed` and no
jobs of further stage are executed.
There are also two edge cases worth mentioning:
1. If no `stages` are defined in `.gitlab-ci.yml`, then the `build`,
`test` and `deploy` are allowed to be used as job's stage by default.
2. If a job doesn't specify a `stage`, the job is assigned the `test` stage.
### types
> Deprecated, and could be removed in one of the future releases. Use [stages](#stages) instead.
Alias for [stages](#stages).
### variables
> Introduced in GitLab Runner v0.5.0.
GitLab CI allows you to add variables to `.gitlab-ci.yml` that are set in the
job environment. The variables are stored in the Git repository and are meant
to store non-sensitive project configuration, for example:
Integers (as well as strings) are legal both for variable's name and value.
Floats are not legal and cannot be used.
These variables can be later used in all executed commands and scripts.
The YAML-defined variables are also set to all created service containers,
thus allowing to fine tune them. Variables can be also defined on a
[job level](#job-variables).
Except for the user defined variables, there are also the ones set up by the
Runner itself. One example would be `CI_COMMIT_REF_NAME` which has the value of
the branch or tag name for which project is built. Apart from the variables
you can set in `.gitlab-ci.yml`, there are also the so called secret variables
which can be set in GitLab's UI.
[Learn more about variables.][variables]
### cache
>
**Notes:**
- Introduced in GitLab Runner v0.7.0.
- Prior to GitLab 9.2, caches were restored after artifacts.
- From GitLab 9.2, caches are restored before artifacts.
`cache` is used to specify a list of files and directories which should be
cached between jobs. You can only use paths that are within the project
workspace.
**By default caching is enabled and shared between pipelines and jobs,
starting from GitLab 9.0**
If `cache` is defined outside the scope of jobs, it means it is set
globally and all jobs will use that definition.
Cache all files in `binaries` and `.config`:
```yaml
rspec:
script:test
cache:
paths:
-binaries/
-.config
```
Cache all Git untracked files:
```yaml
rspec:
script:test
cache:
untracked:true
```
Cache all Git untracked files and files in `binaries`:
```yaml
rspec:
script:test
cache:
untracked:true
paths:
-binaries/
```
Locally defined cache overrides globally defined options. The following `rspec`
job will cache only `binaries/`:
```yaml
cache:
paths:
-my/files
rspec:
script:test
cache:
key:rspec
paths:
-binaries/
```
Note that since cache is shared between jobs, if you're using different
paths for different jobs, you should also set a different **cache:key**
otherwise cache content can be overwritten.
The cache is provided on a best-effort basis, so don't expect that the cache
will be always present. For implementation details, please check GitLab Runner.
#### cache:key
> Introduced in GitLab Runner v1.0.0.
The `key` directive allows you to define the affinity of caching
between jobs, allowing to have a single cache for all jobs,
cache per-job, cache per-branch or any other way you deem proper.
This allows you to fine tune caching, allowing you to cache data between
different jobs or even different branches.
The `cache:key` variable can use any of the [predefined variables](../variables/README.md).
The default key is **default** across the project, therefore everything is
shared between each pipelines and jobs by default, starting from GitLab 9.0.
>**Note:** The `cache:key` variable cannot contain the `/` character, or the equivalent URI encoded `%2F`; a value made only of dots (`.`, `%2E`) is also forbidden.
---
**Example configurations**
To enable per-job caching:
```yaml
cache:
key:"$CI_JOB_NAME"
untracked:true
```
To enable per-branch caching:
```yaml
cache:
key:"$CI_COMMIT_REF_SLUG"
untracked:true
```
To enable per-job and per-branch caching:
```yaml
cache:
key:"$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
untracked:true
```
To enable per-branch and per-stage caching:
```yaml
cache:
key:"$CI_JOB_STAGE-$CI_COMMIT_REF_SLUG"
untracked:true
```
If you use **Windows Batch** to run your shell scripts you need to replace
`$` with `%`:
```yaml
cache:
key:"%CI_JOB_STAGE%-%CI_COMMIT_REF_SLUG%"
untracked:true
```
If you use **Windows PowerShell** to run your shell scripts you need to replace
`$` with `$env:`:
```yaml
cache:
key:"$env:CI_JOB_STAGE-$env:CI_COMMIT_REF_SLUG"
untracked:true
```
### cache:policy
> Introduced in GitLab 9.4.
The default behaviour of a caching job is to download the files at the start of
execution, and to re-upload them at the end. This allows any changes made by the
job to be persisted for future runs, and is known as the `pull-push` cache
policy.
If you know the job doesn't alter the cached files, you can skip the upload step
by setting `policy: pull` in the job specification. Typically, this would be
twinned with an ordinary cache job at an earlier stage to ensure the cache
is updated from time to time:
```yaml
stages:
-setup
-test
prepare:
stage:setup
cache:
key:gems
paths:
-vendor/bundle
script:
-bundle install --deployment
rspec:
stage:test
cache:
key:gems
paths:
-vendor/bundle
policy:pull
script:
-bundle exec rspec ...
```
jobs of further stage are executed.
This helps to speed up job execution and reduce load on the cache server,
especially when you have a large number of cache-using jobs executing in
parallel.
There are also two edge cases worth mentioning:
Additionally, if you have a job that unconditionally recreates the cache without
reference to its previous contents, you can use `policy: push` in that job to
skip the download step.
1. If no `stages` are defined in `.gitlab-ci.yml`, then the `build`,
`test` and `deploy` are allowed to be used as job's stage by default.
2. If a job doesn't specify a `stage`, the job is assigned the `test` stage.
## Jobs
## `stage`
`.gitlab-ci.yml` allows you to specify an unlimited number of jobs. Each job
must have a unique name, which is not one of the keywords mentioned above.
A job is defined by a list of parameters that define the job behavior.
`stage` is defined per-job and relies on [`stages`](#stages) which is defined
globally. It allows to group jobs into different stages, and jobs of the same
`stage` are executed in `parallel`. For example:
```yaml
job_name:
script:
-rake spec
-coverage
stages:
-build
-test
-deploy
job 1:
stage:build
script:make build dependencies
job 2:
stage:build
script:make build artifacts
job 3:
stage:test
only:
-master
except:
-develop
tags:
-ruby
-postgres
allow_failure:true
script:make test
job 4:
stage:deploy
script:make deploy
```
| Keyword | Required | Description |
|---------------|----------|-------------|
| script | yes | Defines a shell script which is executed by Runner |
| image | no | Use docker image, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) |
| services | no | Use docker services, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) |
| stage | no | Defines a job stage (default: `test`) |
| type | no | Alias for `stage` |
| variables | no | Define job variables on a job level |
| only | no | Defines a list of git refs for which job is created |
| except | no | Defines a list of git refs for which job is not created |
| tags | no | Defines a list of tags which are used to select Runner |
| allow_failure | no | Allow job to fail. Failed job doesn't contribute to commit status |
| when | no | Define when to run job. Can be `on_success`, `on_failure`, `always` or `manual` |
| dependencies | no | Define other jobs that a job depends on so that you can pass artifacts between them|
| artifacts | no | Define list of [job artifacts](../../user/project/pipelines/job_artifacts.md) |
| cache | no | Define list of files that should be cached between subsequent runs |
| before_script | no | Override a set of commands that are executed before job |
| after_script | no | Override a set of commands that are executed after job |
| environment | no | Defines a name of environment to which deployment is done by this job |
| coverage | no | Define code coverage settings for a given job |
| retry | no | Define how many times a job can be auto-retried in case of a failure |
## `types`
### script
CAUTION: **Deprecated:**
`types` is deprecated, and could be removed in one of the future releases.
Use [stages](#stages) instead.
`script` is a shell script which is executed by the Runner. For example:
## `script`
`script` is the only required keyword that a job needs. It's a shell script
which is executed by the Runner. For example:
```yaml
job:
...
...
@@ -429,13 +231,7 @@ that the YAML parser knows to interpret the whole thing as a string rather than
a "key: value" pair. Be careful when using special characters:
