@@ -221,8 +221,8 @@ check the value of the `$CI_PIPELINE_SOURCE` variable:
| `external` | When you use CI services other than GitLab. |
| `external_pull_request_event` | When an external pull request on GitHub is created or updated. See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). |
| `merge_request_event` | For pipelines created when a merge request is created or updated. Required to enable [merge request pipelines](../pipelines/merge_request_pipelines.md), [merged results pipelines](../pipelines/pipelines_for_merged_results.md), and [merge trains](../pipelines/merge_trains.md). |
| `parent_pipeline` | For pipelines triggered by a [parent/child pipeline](../parent_child_pipelines.md) with `rules`. Use this pipeline source in the child pipeline configuration so that it can be triggered by the parent pipeline. |
| `pipeline` | For [multi-project pipelines](../multi_project_pipelines.md) created by [using the API with `CI_JOB_TOKEN`](../multi_project_pipelines.md#create-multi-project-pipelines-by-using-the-api), or the [`trigger`](../yaml/index.md#trigger) keyword. |
| `parent_pipeline` | For pipelines triggered by a [parent/child pipeline](../pipelines/parent_child_pipelines.md) with `rules`. Use this pipeline source in the child pipeline configuration so that it can be triggered by the parent pipeline. |
| `pipeline` | For [multi-project pipelines](../pipelines/multi_project_pipelines.md) created by [using the API with `CI_JOB_TOKEN`](../pipelines/multi_project_pipelines.md#create-multi-project-pipelines-by-using-the-api), or the [`trigger`](../yaml/index.md#trigger) keyword. |
| `push` | For pipelines triggered by a `git push` event, including for branches and tags. |
| `schedule` | For [scheduled pipelines](../pipelines/schedules.md). |
| `trigger` | For pipelines created by using a [trigger token](../triggers/index.md#trigger-token). |
info:To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
type:reference
redirect_to:'pipelines/parent_child_pipelines.md'
---
# Parent-child pipelines **(FREE)**
This document was moved to [another location](pipelines/parent_child_pipelines.md).
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16094) in GitLab 12.7.
As pipelines grow more complex, a few related problems start to emerge:
- The staged structure, where all steps in a stage must be completed before the first
job in next stage begins, causes arbitrary waits, slowing things down.
- Configuration for the single global pipeline becomes very long and complicated,
making it hard to manage.
- Imports with [`include`](yaml/index.md#include) increase the complexity of the configuration, and create the potential
for namespace collisions where jobs are unintentionally duplicated.
- Pipeline UX can become unwieldy with so many jobs and stages to work with.
Additionally, sometimes the behavior of a pipeline needs to be more dynamic. The ability
to choose to start sub-pipelines (or not) is a powerful ability, especially if the
For an overview, see [Parent-Child Pipelines feature demo](https://youtu.be/n8KpBSqZNbk).
## Examples
The simplest case is [triggering a child pipeline](yaml/index.md#trigger) using a
local YAML file to define the pipeline configuration. In this case, the parent pipeline
triggers the child pipeline, and continues without waiting:
```yaml
microservice_a:
trigger:
include:path/to/microservice_a.yml
```
You can include multiple files when composing a child pipeline:
```yaml
microservice_a:
trigger:
include:
-local:path/to/microservice_a.yml
-template:Security/SAST.gitlab-ci.yml
```
In [GitLab 13.5](https://gitlab.com/gitlab-org/gitlab/-/issues/205157) and later,
you can use [`include:file`](yaml/index.md#includefile) to trigger child pipelines
with a configuration file in a different project:
```yaml
microservice_a:
trigger:
include:
-project:'my-group/my-pipeline-library'
file:'path/to/ci-config.yml'
```
The maximum number of entries that are accepted for `trigger:include:` is three.
Similar to [multi-project pipelines](multi_project_pipelines.md#mirror-status-of-a-triggered-pipeline-in-the-trigger-job),
we can set the parent pipeline to depend on the status of the child pipeline upon completion:
```yaml
microservice_a:
trigger:
include:
-local:path/to/microservice_a.yml
-template:Security/SAST.gitlab-ci.yml
strategy:depend
```
## Merge Request child pipelines
To trigger a child pipeline as a [Merge Request Pipeline](pipelines/merge_request_pipelines.md) we need to:
- Set the trigger job to run on merge requests:
```yaml
# parent .gitlab-ci.yml
microservice_a:
trigger:
include:path/to/microservice_a.yml
rules:
-if:$CI_MERGE_REQUEST_ID
```
- Configure the child pipeline by either:
- Setting all jobs in the child pipeline to evaluate in the context of a merge request:
```yaml
# child path/to/microservice_a.yml
workflow:
rules:
- if: $CI_MERGE_REQUEST_ID
job1:
script: ...
job2:
script: ...
```
- Alternatively, setting the rule per job. For example, to create only `job1` in
the context of merge request pipelines:
```yaml
# child path/to/microservice_a.yml
job1:
script: ...
rules:
- if: $CI_MERGE_REQUEST_ID
job2:
script: ...
```
## Dynamic child pipelines
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35632) in GitLab 12.9.
Instead of running a child pipeline from a static YAML file, you can define a job that runs
your own script to generate a YAML file, which is then [used to trigger a child pipeline](yaml/index.md#trigger-child-pipeline-with-generated-configuration-file).
This technique can be very powerful in generating pipelines targeting content that changed or to
For an overview, see [Create child pipelines using dynamically generated configurations](https://youtu.be/nMdfus2JWHM).
<!-- vale gitlab.Spelling = NO -->
We also have an example project using
[Dynamic Child Pipelines with Jsonnet](https://gitlab.com/gitlab-org/project-templates/jsonnet)
which shows how to use a data templating language to generate your `.gitlab-ci.yml` at runtime. You could use a similar process for other templating languages like [Dhall](https://dhall-lang.org/) or [`ytt`](https://get-ytt.io/).
<!-- vale gitlab.Spelling = NO -->
The artifact path is parsed by GitLab, not the runner, so the path must match the
syntax for the OS running GitLab. If GitLab is running on Linux but using a Windows
runner for testing, the path separator for the trigger job would be `/`. Other CI/CD
configuration for jobs, like scripts, that use the Windows runner would use `\`.
In GitLab 12.9, the child pipeline could fail to be created in certain cases, causing the parent pipeline to fail.
This is [resolved in GitLab 12.10](https://gitlab.com/gitlab-org/gitlab/-/issues/209070).
## Nested child pipelines
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29651) in GitLab 13.4.
> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/243747) in GitLab 13.5.
Parent and child pipelines were introduced with a maximum depth of one level of child
pipelines, which was later increased to two. A parent pipeline can trigger many child
pipelines, and these child pipelines can trigger their own child pipelines. It's not
possible to trigger another level of child pipelines.
info:To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
type:reference
---
# Parent-child pipelines **(FREE)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16094) in GitLab 12.7.
As pipelines grow more complex, a few related problems start to emerge:
- The staged structure, where all steps in a stage must be completed before the first
job in next stage begins, causes arbitrary waits, slowing things down.
- Configuration for the single global pipeline becomes very long and complicated,
making it hard to manage.
- Imports with [`include`](../yaml/index.md#include) increase the complexity of the configuration, and create the potential
for namespace collisions where jobs are unintentionally duplicated.
- Pipeline UX can become unwieldy with so many jobs and stages to work with.
Additionally, sometimes the behavior of a pipeline needs to be more dynamic. The ability
to choose to start sub-pipelines (or not) is a powerful ability, especially if the
For an overview, see [Parent-Child Pipelines feature demo](https://youtu.be/n8KpBSqZNbk).
## Examples
The simplest case is [triggering a child pipeline](../yaml/index.md#trigger) using a
local YAML file to define the pipeline configuration. In this case, the parent pipeline
triggers the child pipeline, and continues without waiting:
```yaml
microservice_a:
trigger:
include:path/to/microservice_a.yml
```
You can include multiple files when composing a child pipeline:
```yaml
microservice_a:
trigger:
include:
-local:path/to/microservice_a.yml
-template:Security/SAST.gitlab-ci.yml
```
In [GitLab 13.5](https://gitlab.com/gitlab-org/gitlab/-/issues/205157) and later,
you can use [`include:file`](../yaml/index.md#includefile) to trigger child pipelines
with a configuration file in a different project:
```yaml
microservice_a:
trigger:
include:
-project:'my-group/my-pipeline-library'
file:'path/to/ci-config.yml'
```
The maximum number of entries that are accepted for `trigger:include:` is three.
Similar to [multi-project pipelines](multi_project_pipelines.md#mirror-status-of-a-triggered-pipeline-in-the-trigger-job),
we can set the parent pipeline to depend on the status of the child pipeline upon completion:
```yaml
microservice_a:
trigger:
include:
-local:path/to/microservice_a.yml
-template:Security/SAST.gitlab-ci.yml
strategy:depend
```
## Merge Request child pipelines
To trigger a child pipeline as a [Merge Request Pipeline](merge_request_pipelines.md) we need to:
- Set the trigger job to run on merge requests:
```yaml
# parent .gitlab-ci.yml
microservice_a:
trigger:
include:path/to/microservice_a.yml
rules:
-if:$CI_MERGE_REQUEST_ID
```
- Configure the child pipeline by either:
- Setting all jobs in the child pipeline to evaluate in the context of a merge request:
```yaml
# child path/to/microservice_a.yml
workflow:
rules:
- if: $CI_MERGE_REQUEST_ID
job1:
script: ...
job2:
script: ...
```
- Alternatively, setting the rule per job. For example, to create only `job1` in
the context of merge request pipelines:
```yaml
# child path/to/microservice_a.yml
job1:
script: ...
rules:
- if: $CI_MERGE_REQUEST_ID
job2:
script: ...
```
## Dynamic child pipelines
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35632) in GitLab 12.9.
Instead of running a child pipeline from a static YAML file, you can define a job that runs
your own script to generate a YAML file, which is then [used to trigger a child pipeline](../yaml/index.md#trigger-child-pipeline-with-generated-configuration-file).
This technique can be very powerful in generating pipelines targeting content that changed or to
For an overview, see [Create child pipelines using dynamically generated configurations](https://youtu.be/nMdfus2JWHM).
<!-- vale gitlab.Spelling = NO -->
We also have an example project using
[Dynamic Child Pipelines with Jsonnet](https://gitlab.com/gitlab-org/project-templates/jsonnet)
which shows how to use a data templating language to generate your `.gitlab-ci.yml` at runtime. You could use a similar process for other templating languages like [Dhall](https://dhall-lang.org/) or [`ytt`](https://get-ytt.io/).
<!-- vale gitlab.Spelling = NO -->
The artifact path is parsed by GitLab, not the runner, so the path must match the
syntax for the OS running GitLab. If GitLab is running on Linux but using a Windows
runner for testing, the path separator for the trigger job would be `/`. Other CI/CD
configuration for jobs, like scripts, that use the Windows runner would use `\`.
In GitLab 12.9, the child pipeline could fail to be created in certain cases, causing the parent pipeline to fail.
This is [resolved in GitLab 12.10](https://gitlab.com/gitlab-org/gitlab/-/issues/209070).
## Nested child pipelines
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29651) in GitLab 13.4.
> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/243747) in GitLab 13.5.
Parent and child pipelines were introduced with a maximum depth of one level of child
pipelines, which was later increased to two. A parent pipeline can trigger many child
pipelines, and these child pipelines can trigger their own child pipelines. It's not
possible to trigger another level of child pipelines.
@@ -581,8 +581,8 @@ You can override the value of a variable when you:
1. Run a job manually in the UI.
1. Use [push options](../../user/project/push_options.md#push-options-for-gitlab-cicd).
1. Trigger a pipeline by using [the API](../triggers/index.md#making-use-of-trigger-variables).
1. Pass variables to a downstream pipeline [by using the `variable` keyword](../multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-the-variables-keyword)
or [by using variable inheritance](../multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-variable-inheritance).
1. Pass variables to a downstream pipeline [by using the `variable` keyword](../pipelines/multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-the-variables-keyword)
or [by using variable inheritance](../pipelines/multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-variable-inheritance).
The pipeline variables declared in these events take [priority over other variables](#cicd-variable-precedence).
@@ -1336,7 +1336,7 @@ pipeline based on branch names or pipeline types.
| `external` | When you use CI services other than GitLab. |
| `external_pull_requests` | When an external pull request on GitHub is created or updated (See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests)). |
| `merge_requests` | For pipelines created when a merge request is created or updated. Enables [merge request pipelines](../pipelines/merge_request_pipelines.md), [merged results pipelines](../pipelines/pipelines_for_merged_results.md), and [merge trains](../pipelines/merge_trains.md). |
| `pipelines` | For [multi-project pipelines](../multi_project_pipelines.md) created by [using the API with `CI_JOB_TOKEN`](../multi_project_pipelines.md#create-multi-project-pipelines-by-using-the-api), or the [`trigger`](#trigger) keyword. |
| `pipelines` | For [multi-project pipelines](../pipelines/multi_project_pipelines.md) created by [using the API with `CI_JOB_TOKEN`](../pipelines/multi_project_pipelines.md#create-multi-project-pipelines-by-using-the-api), or the [`trigger`](#trigger) keyword. |
| `pushes` | For pipelines triggered by a `git push` event, including for branches and tags. |
| `schedules` | For [scheduled pipelines](../pipelines/schedules.md). |
| `tags` | When the Git reference for a pipeline is a tag. |
...
...
@@ -1710,7 +1710,7 @@ build_job:
You can't download artifacts from jobs that run in [`parallel:`](#parallel).
To download artifacts between [parent-child pipelines](../parent_child_pipelines.md),
To download artifacts between [parent-child pipelines](../pipelines/parent_child_pipelines.md),
use [`needs:pipeline`](#artifact-downloads-to-child-pipelines).
You should not download artifacts from the same ref as a running pipeline. Concurrent
...
...
@@ -1720,7 +1720,7 @@ pipelines running on the same ref could override the artifacts.
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/255983) in GitLab v13.7.
A [child pipeline](../parent_child_pipelines.md) can download artifacts from a job in
A [child pipeline](../pipelines/parent_child_pipelines.md) can download artifacts from a job in
its parent pipeline or another child pipeline in the same parent-child pipeline hierarchy.
For example, with the following parent pipeline that has a job that creates some artifacts:
...
...
@@ -3305,7 +3305,7 @@ If there is more than one matched line in the job output, the last line is used.
For the matched line, the first occurrence of `\d+(\.\d+)?` is the code coverage.
Leading zeros are removed.
Coverage output from [child pipelines](../parent_child_pipelines.md) is not recorded
Coverage output from [child pipelines](../pipelines/parent_child_pipelines.md) is not recorded
or displayed. Check [the related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/280818)
Use `trigger` to define a downstream pipeline trigger. When GitLab starts a `trigger` job,
a downstream pipeline is created.
Jobs with `trigger` can only use a [limited set of keywords](../multi_project_pipelines.md#define-multi-project-pipelines-in-your-gitlab-ciyml-file).
Jobs with `trigger` can only use a [limited set of keywords](../pipelines/multi_project_pipelines.md#define-multi-project-pipelines-in-your-gitlab-ciyml-file).
For example, you can't run commands with [`script`](#script), [`before_script`](#before_script),
or [`after_script`](#after_script).
You can use this keyword to create two different types of downstream pipelines:
[In GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/197140/) and later, you can
view which job triggered a downstream pipeline. In the [pipeline graph](../pipelines/index.md#visualize-pipelines),
...
...
@@ -3633,7 +3633,7 @@ upstream_bridge:
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16094) in GitLab 12.7.
To create a [child pipeline](../parent_child_pipelines.md), specify the path to the
To create a [child pipeline](../pipelines/parent_child_pipelines.md), specify the path to the
YAML file that contains the configuration of the child pipeline:
```yaml
...
...
@@ -3642,7 +3642,7 @@ trigger_job:
include:path/to/child-pipeline.yml
```
Similar to [multi-project pipelines](../multi_project_pipelines.md#mirror-status-of-a-triggered-pipeline-in-the-trigger-job),
Similar to [multi-project pipelines](../pipelines/multi_project_pipelines.md#mirror-status-of-a-triggered-pipeline-in-the-trigger-job),
it's possible to mirror the status from a triggered pipeline:
```yaml
...
...
@@ -3657,7 +3657,7 @@ trigger_job:
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35632) in GitLab 12.9.
You can also trigger a child pipeline from a [dynamically generated configuration file](../parent_child_pipelines.md#dynamic-child-pipelines):
You can also trigger a child pipeline from a [dynamically generated configuration file](../pipelines/parent_child_pipelines.md#dynamic-child-pipelines):
@@ -30,7 +30,7 @@ On the left side we have the events that can trigger a pipeline based on various
- When a [merge request is created or updated](../../ci/pipelines/merge_request_pipelines.md#pipelines-for-merge-requests).
- When an MR is added to a [Merge Train](../../ci/pipelines/merge_trains.md#merge-trains).
- A [scheduled pipeline](../../ci/pipelines/schedules.md#pipeline-schedules).
- When project is [subscribed to an upstream project](../../ci/multi_project_pipelines.md#trigger-a-pipeline-when-an-upstream-project-is-rebuilt).
- When project is [subscribed to an upstream project](../../ci/pipelines/multi_project_pipelines.md#trigger-a-pipeline-when-an-upstream-project-is-rebuilt).
- When [Auto DevOps](../../topics/autodevops/index.md) is enabled.
- When GitHub integration is used with [external pull requests](../../ci/ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests).
- When an upstream pipeline contains a [bridge job](../../ci/yaml/index.md#trigger) which triggers a downstream pipeline.
It's also possible to run the fuzzing jobs longer and without blocking your main pipeline. This
configuration uses the GitLab [parent-child pipelines](../../../ci/parent_child_pipelines.md).
configuration uses the GitLab [parent-child pipelines](../../../ci/pipelines/parent_child_pipelines.md).
The full example is available in the [repository](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example/-/tree/continuous_fuzzing#running-go-fuzz-from-ci).
This example uses Go, but is applicable for any other supported languages.
@@ -64,7 +64,7 @@ With GitLab Enterprise Edition, you can also:
- Leverage [Elasticsearch](../integration/elasticsearch.md) with [Advanced Search](search/advanced_search.md) for faster, more advanced code search across your entire GitLab instance.
-[Authenticate users with Kerberos](../integration/kerberos.md).
-[Mirror a repository](project/repository/repository_mirroring.md) from elsewhere on your local server.
- View your entire CI/CD pipeline involving more than one project with [Multiple-Project Pipelines](../ci/multi_project_pipelines.md).
- View your entire CI/CD pipeline involving more than one project with [Multiple-Project Pipelines](../ci/pipelines/multi_project_pipelines.md).
-[Lock files](project/file_lock.md) to prevent conflicts.
- View the current health and status of each CI environment running on Kubernetes with [Deploy Boards](project/deploy_boards.md).
- Leverage continuous delivery method with [Canary Deployments](project/canary_deployments.md).
@@ -22,7 +22,7 @@ or link to useful information directly from merge requests:
| [Unit test reports](../../../ci/unit_test_reports.md) | Configure your CI jobs to use Unit test reports, and let GitLab display a report on the merge request so that it's easier and faster to identify the failure without having to check the entire job log. |
| [License Compliance](../../compliance/license_compliance/index.md)**(ULTIMATE)** | Manage the licenses of your dependencies. |
| [Metrics Reports](../../../ci/metrics_reports.md)**(PREMIUM)** | Display the Metrics Report on the merge request so that it's fast and easy to identify changes to important metrics. |
| [Multi-Project pipelines](../../../ci/multi_project_pipelines.md)**(PREMIUM)** | When you set up GitLab CI/CD across multiple projects, you can visualize the entire pipeline, including all cross-project interdependencies. |
| [Multi-Project pipelines](../../../ci/pipelines/multi_project_pipelines.md)**(PREMIUM)** | When you set up GitLab CI/CD across multiple projects, you can visualize the entire pipeline, including all cross-project interdependencies. |
| [Pipelines for merge requests](../../../ci/pipelines/merge_request_pipelines.md) | Customize a specific pipeline structure for merge requests in order to speed the cycle up by running only important jobs. |
| [Pipeline Graphs](../../../ci/pipelines/index.md#visualize-pipelines) | View the status of pipelines within the merge request, including the deployment process. |
| [Test Coverage visualization](test_coverage_visualization.md) | See test coverage results for merge requests, within the file diff. |
