1.[Trigger variables][triggers] or [scheduled pipeline variables](../../user/project/pipelines/schedules.md#making-use-of-scheduled-pipeline-variables)(take precedence over all)
1.[Trigger variables][triggers] or [scheduled pipeline variables](../../user/project/pipelines/schedules.md#making-use-of-scheduled-pipeline-variables)(take precedence over all)
1. Project-level [secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables)
1. Project-level [secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables)
1. Group-level [secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables)
1. Group-level [secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables)
This document describes the usage of `.gitlab-ci.yml`, the file that is used by
This document describes the usage of `.gitlab-ci.yml`, the file that is used by
GitLab Runner to manage your project's jobs.
GitLab Runner to manage your project's jobs.
If you want a quick introduction to GitLab CI, follow our
[quick start guide](../quick_start/README.md).
## .gitlab-ci.yml
From version 7.12, GitLab CI uses a [YAML](https://en.wikipedia.org/wiki/YAML)
From version 7.12, GitLab CI uses a [YAML](https://en.wikipedia.org/wiki/YAML)
file (`.gitlab-ci.yml`) for the project configuration. It is placed in the root
file (`.gitlab-ci.yml`) for the project configuration. It is placed in the root
of your repository and contains definitions of how your project should be built.
of your repository and contains definitions of how your project should be built.
If you want a quick introduction to GitLab CI, follow our
[quick start guide](../quick_start/README.md).
## Jobs
The YAML file defines a set of jobs with constraints stating when they should
The YAML file defines a set of jobs with constraints stating when they should
be run. The jobs are defined as top-level elements with a name and always have
be run. You can specify an unlimited number of jobs which are defined as
to contain at least the `script` clause:
top-level elements with an arbitrary name and always have to contain at least
the `script` clause.
```yaml
```yaml
job1:
job1:
...
@@ -24,9 +25,8 @@ job2:
...
@@ -24,9 +25,8 @@ job2:
script:"execute-script-for-job2"
script:"execute-script-for-job2"
```
```
The above example is the simplest possible CI configuration with two separate
The above example is the simplest possible CI/CD configuration with two separate
jobs, where each of the jobs executes a different command.
jobs, where each of the jobs executes a different command.
Of course a command can execute code directly (`./configure;make;make install`)
Of course a command can execute code directly (`./configure;make;make install`)
or run a script (`test.sh`) in the repository.
or run a script (`test.sh`) in the repository.
...
@@ -34,78 +34,115 @@ Jobs are picked up by [Runners](../runners/README.md) and executed within the
...
@@ -34,78 +34,115 @@ Jobs are picked up by [Runners](../runners/README.md) and executed within the
environment of the Runner. What is important, is that each job is run
environment of the Runner. What is important, is that each job is run
independently from each other.
independently from each other.
The YAML syntax allows for using more complex job specifications than in the
Each job must have a unique name, but there are a few **reserved `keywords` that
above example:
cannot be used as job names**:
```yaml
-`image`
image:ruby:2.1
-`services`
services:
-`stages`
-postgres
-`types`
-`before_script`
-`after_script`
-`variables`
-`cache`
before_script:
A job is defined by a list of parameters that define the job behavior.
-bundle install
after_script:
| Keyword | Required | Description |
-rm secrets
|---------------|----------|-------------|
| script | yes | Defines a shell script which is executed by Runner |
| image | no | Use docker image, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) |
| services | no | Use docker services, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) |
| stage | no | Defines a job stage (default: `test`) |
| type | no | Alias for `stage` |
| variables | no | Define job variables on a job level |
| only | no | Defines a list of git refs for which job is created |
| except | no | Defines a list of git refs for which job is not created |
| tags | no | Defines a list of tags which are used to select Runner |
| allow_failure | no | Allow job to fail. Failed job doesn't contribute to commit status |
| when | no | Define when to run job. Can be `on_success`, `on_failure`, `always` or `manual` |
| dependencies | no | Define other jobs that a job depends on so that you can pass artifacts between them|
| artifacts | no | Define list of [job artifacts](#artifacts) |
| cache | no | Define list of files that should be cached between subsequent runs |
| before_script | no | Override a set of commands that are executed before job |
| after_script | no | Override a set of commands that are executed after job |
| environment | no | Defines a name of environment to which deployment is done by this job |
| coverage | no | Define code coverage settings for a given job |
| retry | no | Define how many times a job can be auto-retried in case of a failure |
stages:
### `pages`
-build
-test
-deploy
job1:
`pages` is a special job that is used to upload static content to GitLab that
stage:build
can be used to serve your website. It has a special syntax, so the two
requirements below must be met:
1. Any static content must be placed under a `public/` directory
1.`artifacts` with a path to the `public/` directory must be defined
The example below simply moves all files from the root of the project to the
`public/` directory. The `.public` workaround is so `cp` doesn't also copy
`public/` to itself in an infinite loop:
```
pages:
stage: deploy
script:
script:
-execute-script-for-job1
- mkdir .public
- cp -r * .public
- mv .public public
artifacts:
paths:
- public
only:
only:
-master
- master
tags:
-docker
```
```
There are a few reserved `keywords` that **cannot** be used as job names:
Read more on [GitLab Pages user documentation](../../user/project/pages/index.md).
| Keyword | Required | Description |
|---------------|----------|-------------|
| image | no | Use docker image, covered in [Use Docker](../docker/README.md) |
| services | no | Use docker services, covered in [Use Docker](../docker/README.md) |
| stages | no | Define build stages |
| types | no | Alias for `stages` (deprecated) |
| before_script | no | Define commands that run before each job's script |
| after_script | no | Define commands that run after each job's script |
| variables | no | Define build variables |
| cache | no | Define list of files that should be cached between subsequent runs |
### image and services
## `image` and `services`
This allows to specify a custom Docker image and a list of services that can be
This allows to specify a custom Docker image and a list of services that can be
used for time of the job. The configuration of this feature is covered in
used for time of the job. The configuration of this feature is covered in
[a separate document](../docker/README.md).
[a separate document](../docker/README.md).
### before_script
## `before_script` and `after_script`
`before_script` is used to define the command that should be run before all
jobs, including deploy jobs, but after the restoration of artifacts. This can
be an array or a multi-line string.
### after_script
> Introduced in GitLab 8.7 and requires Gitlab Runner v1.2
> Introduced in GitLab 8.7 and requires Gitlab Runner v1.2
`before_script` is used to define the command that should be run before all
jobs, including deploy jobs, but after the restoration of [artifacts](#artifacts).
This can be an array or a multi-line string.
`after_script` is used to define the command that will be run after for all
`after_script` is used to define the command that will be run after for all
jobs, including failed ones. This has to be an array or a multi-line string.
jobs, including failed ones. This has to be an array or a multi-line string.
> **Note:**
The `before_script` and the main `script` are concatenated and run in a single context/container.
The `before_script` and the main `script` are concatenated and run in a single context/container.
The `after_script` is run separately, so depending on the executor, changes done
The `after_script` is run separately, so depending on the executor, changes done
outside of the working tree might not be visible, e.g. software installed in the
outside of the working tree might not be visible, e.g. software installed in the
`before_script`.
`before_script`.
### stages
It's possible to overwrite the globally defined `before_script` and `after_script`
if you set it per-job:
`stages` is used to define stages that can be used by jobs.
```yaml
The specification of `stages` allows for having flexible multi stage pipelines.
before_script:
-global before script
job:
before_script:
-execute this instead of global before script
script:
-my command
after_script:
-execute this after my script
```
## `stages`
`stages` is used to define stages that can be used by jobs and is defined
globally.
The specification of `stages` allows for having flexible multi stage pipelines.
The ordering of elements in `stages` defines the ordering of jobs' execution:
The ordering of elements in `stages` defines the ordering of jobs' execution:
1. Jobs of the same stage are run in parallel.
1. Jobs of the same stage are run in parallel.
...
@@ -125,289 +162,54 @@ stages:
...
@@ -125,289 +162,54 @@ stages:
1. If all jobs of `build` succeed, the `test` jobs are executed in parallel.
1. If all jobs of `build` succeed, the `test` jobs are executed in parallel.
1. If all jobs of `test` succeed, the `deploy` jobs are executed in parallel.
1. If all jobs of `test` succeed, the `deploy` jobs are executed in parallel.
1. If all jobs of `deploy` succeed, the commit is marked as `passed`.
1. If all jobs of `deploy` succeed, the commit is marked as `passed`.
1. If any of the previous jobs fails, the commit is marked as `failed` and no
1. If any of the previous jobs fails, the commit is marked as `failed` and no
jobs of further stage are executed.
jobs of further stage are executed.
There are also two edge cases worth mentioning:
1. If no `stages` are defined in `.gitlab-ci.yml`, then the `build`,
`test` and `deploy` are allowed to be used as job's stage by default.
2. If a job doesn't specify a `stage`, the job is assigned the `test` stage.
### types
> Deprecated, and could be removed in one of the future releases. Use [stages](#stages) instead.
Alias for [stages](#stages).
### variables
> Introduced in GitLab Runner v0.5.0.
GitLab CI allows you to add variables to `.gitlab-ci.yml` that are set in the
job environment. The variables are stored in the Git repository and are meant
to store non-sensitive project configuration, for example:
Integers (as well as strings) are legal both for variable's name and value.
Floats are not legal and cannot be used.
These variables can be later used in all executed commands and scripts.
The YAML-defined variables are also set to all created service containers,
thus allowing to fine tune them. Variables can be also defined on a
[job level](#job-variables).
Except for the user defined variables, there are also the ones set up by the
Runner itself. One example would be `CI_COMMIT_REF_NAME` which has the value of
the branch or tag name for which project is built. Apart from the variables
you can set in `.gitlab-ci.yml`, there are also the so called secret variables
which can be set in GitLab's UI.
[Learn more about variables.][variables]
### cache
>
**Notes:**
- Introduced in GitLab Runner v0.7.0.
- Prior to GitLab 9.2, caches were restored after artifacts.
- From GitLab 9.2, caches are restored before artifacts.
`cache` is used to specify a list of files and directories which should be
cached between jobs. You can only use paths that are within the project
workspace.
**By default caching is enabled and shared between pipelines and jobs,
starting from GitLab 9.0**
If `cache` is defined outside the scope of jobs, it means it is set
globally and all jobs will use that definition.
Cache all files in `binaries` and `.config`:
```yaml
rspec:
script:test
cache:
paths:
-binaries/
-.config
```
Cache all Git untracked files:
```yaml
rspec:
script:test
cache:
untracked:true
```
Cache all Git untracked files and files in `binaries`:
```yaml
rspec:
script:test
cache:
untracked:true
paths:
-binaries/
```
Locally defined cache overrides globally defined options. The following `rspec`
job will cache only `binaries/`:
```yaml
cache:
paths:
-my/files
rspec:
script:test
cache:
key:rspec
paths:
-binaries/
```
Note that since cache is shared between jobs, if you're using different
paths for different jobs, you should also set a different **cache:key**
otherwise cache content can be overwritten.
The cache is provided on a best-effort basis, so don't expect that the cache
will be always present. For implementation details, please check GitLab Runner.
#### cache:key
> Introduced in GitLab Runner v1.0.0.
The `key` directive allows you to define the affinity of caching
between jobs, allowing to have a single cache for all jobs,
cache per-job, cache per-branch or any other way you deem proper.
This allows you to fine tune caching, allowing you to cache data between
different jobs or even different branches.
The `cache:key` variable can use any of the [predefined variables](../variables/README.md).
The default key is **default** across the project, therefore everything is
shared between each pipelines and jobs by default, starting from GitLab 9.0.
>**Note:** The `cache:key` variable cannot contain the `/` character, or the equivalent URI encoded `%2F`; a value made only of dots (`.`, `%2E`) is also forbidden.
---
**Example configurations**
To enable per-job caching:
```yaml
cache:
key:"$CI_JOB_NAME"
untracked:true
```
To enable per-branch caching:
```yaml
cache:
key:"$CI_COMMIT_REF_SLUG"
untracked:true
```
To enable per-job and per-branch caching:
```yaml
cache:
key:"$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
untracked:true
```
To enable per-branch and per-stage caching:
```yaml
cache:
key:"$CI_JOB_STAGE-$CI_COMMIT_REF_SLUG"
untracked:true
```
If you use **Windows Batch** to run your shell scripts you need to replace
`$` with `%`:
```yaml
cache:
key:"%CI_JOB_STAGE%-%CI_COMMIT_REF_SLUG%"
untracked:true
```
If you use **Windows PowerShell** to run your shell scripts you need to replace
`$` with `$env:`:
```yaml
cache:
key:"$env:CI_JOB_STAGE-$env:CI_COMMIT_REF_SLUG"
untracked:true
```
### cache:policy
> Introduced in GitLab 9.4.
The default behaviour of a caching job is to download the files at the start of
execution, and to re-upload them at the end. This allows any changes made by the
job to be persisted for future runs, and is known as the `pull-push` cache
policy.
If you know the job doesn't alter the cached files, you can skip the upload step
by setting `policy: pull` in the job specification. Typically, this would be
twinned with an ordinary cache job at an earlier stage to ensure the cache
is updated from time to time:
```yaml
stages:
-setup
-test
prepare:
stage:setup
cache:
key:gems
paths:
-vendor/bundle
script:
-bundle install --deployment
rspec:
stage:test
cache:
key:gems
paths:
-vendor/bundle
policy:pull
script:
-bundle exec rspec ...
```
This helps to speed up job execution and reduce load on the cache server,
There are also two edge cases worth mentioning:
especially when you have a large number of cache-using jobs executing in
parallel.
Additionally, if you have a job that unconditionally recreates the cache without
1. If no `stages` are defined in `.gitlab-ci.yml`, then the `build`,
reference to its previous contents, you can use `policy: push` in that job to
`test` and `deploy` are allowed to be used as job's stage by default.
skip the download step.
2. If a job doesn't specify a `stage`, the job is assigned the `test` stage.
## Jobs
## `stage`
`.gitlab-ci.yml` allows you to specify an unlimited number of jobs. Each job
`stage` is defined per-job and relies on [`stages`](#stages) which is defined
must have a unique name, which is not one of the keywords mentioned above.
globally. It allows to group jobs into different stages, and jobs of the same
A job is defined by a list of parameters that define the job behavior.
`stage` are executed in `parallel`. For example:
```yaml
```yaml
job_name:
stages:
script:
-build
-rake spec
-test
-coverage
-deploy
job 1:
stage:build
script:make build dependencies
job 2:
stage:build
script:make build artifacts
job 3:
stage:test
stage:test
only:
script:make test
-master
except:
job 4:
-develop
stage:deploy
tags:
script:make deploy
-ruby
-postgres
allow_failure:true
```
```
| Keyword | Required | Description |
## `types`
|---------------|----------|-------------|
| script | yes | Defines a shell script which is executed by Runner |
| image | no | Use docker image, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) |
| services | no | Use docker services, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) |
| stage | no | Defines a job stage (default: `test`) |
| type | no | Alias for `stage` |
| variables | no | Define job variables on a job level |
| only | no | Defines a list of git refs for which job is created |
| except | no | Defines a list of git refs for which job is not created |
| tags | no | Defines a list of tags which are used to select Runner |
| allow_failure | no | Allow job to fail. Failed job doesn't contribute to commit status |
| when | no | Define when to run job. Can be `on_success`, `on_failure`, `always` or `manual` |
| dependencies | no | Define other jobs that a job depends on so that you can pass artifacts between them|
| artifacts | no | Define list of [job artifacts](../../user/project/pipelines/job_artifacts.md) |
| cache | no | Define list of files that should be cached between subsequent runs |
| before_script | no | Override a set of commands that are executed before job |
| after_script | no | Override a set of commands that are executed after job |
| environment | no | Defines a name of environment to which deployment is done by this job |
| coverage | no | Define code coverage settings for a given job |
| retry | no | Define how many times a job can be auto-retried in case of a failure |
### script
CAUTION: **Deprecated:**
`types` is deprecated, and could be removed in one of the future releases.
Use [stages](#stages) instead.
`script` is a shell script which is executed by the Runner. For example:
## `script`
`script` is the only required keyword that a job needs. It's a shell script
which is executed by the Runner. For example:
```yaml
```yaml
job:
job:
...
@@ -429,13 +231,7 @@ that the YAML parser knows to interpret the whole thing as a string rather than
...
@@ -429,13 +231,7 @@ that the YAML parser knows to interpret the whole thing as a string rather than
a "key: value" pair. Be careful when using special characters:
a "key: value" pair. Be careful when using special characters:
