Merge branch 'docs-for-mr-pipelines' into 'master'

Documentation for "`only/except: merge-requests` for merge request pipelines"

Closes #54511 and #15310

See merge request gitlab-org/gitlab-ce!23397

doc/ci/README.md

@@ -71,6 +71,7 @@ learn how to leverage its potential even more.
 - [Caching dependencies](caching/index.md)
 - [Git submodules](git_submodules.md) - How to run your CI jobs when Git
   submodules are involved
+- [Pipelines for merge requests](merge_request_pipelines/index.md)
 - [Use SSH keys in your build environment](ssh_keys/README.md)
 - [Trigger pipelines through the GitLab API](triggers/README.md)
 - [Trigger pipelines on a schedule](../user/project/pipelines/schedules.md)

doc/ci/merge_request_pipelines/index.md

+# Pipelines for merge requests
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/15310) in GitLab 11.6
+
+Usually, when a developer creates a new merge request, a pipeline runs on the
+new change and checks if it's qualified to be merged into a target branch. This
+pipeline should contain only necessary jobs for checking the new changes.
+For example, unit tests, lint checks, and Review Apps are often used in this cycle.
+
+With pipelines for merge requests, you can design a specific pipeline structure
+for merge requests. All you need to do is just adding `only: [merge_requests]` to
+the jobs that you want it to run for only merge requests.
+Every time, when developers create or update merge requests, a pipeline runs on
+their new commits at every push to GitLab.
+
+NOTE: **Note**:
+If you use both this feature and the [Merge When Pipeline Succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md)
+feature, pipelines for merge requests take precendence over the other regular pipelines.
+
+For example, consider a GitLab CI/CD configuration in .gitlab-ci.yml as follows:
+
+```yaml
+build:
+  stage: build
+  script: ./build
+  only:
+  - branches
+  - tags
+  - merge_requests
+
+test:
+  stage: test
+  script: ./test
+  only:
+  - merge_requests
+
+deploy:
+  stage: deploy
+  script: ./deploy
+```
+
+After a developer updated code in a merge request with whatever methods (e.g. `git push`),
+GitLab detects that the code is updated and create a new pipeline for the merge request.
+The pipeline fetches the latest code from the source branch and run tests against it.
+In this example, the pipeline contains only `build` and `test` jobs.
+Since `deploy` job does not have the `only: [merge_requests]` rule,
+deployment jobs will not happen in the merge request.
+
+Consider this pipeline list viewed from the **Pipelines** tab in a merge request:
+
+![Merge request page](img/merge_request.png)
+
+Note that pipelines tagged as **merge request** indicate that they were triggered
+when a merge request was created or updated.
+
+The same tag is shown on the pipeline's details:
+
+![Pipeline's details](img/pipeline_detail.png)
+
+## Important notes about merge requests from forked projects
+
+Note that the current behavior is subject to change. In the usual contribution
+flow, external contributors follow the following steps:
+
+1. Fork a parent project.
+1. Create a merge request from the forked project that targets the `master` branch
+in the parent project.
+1. A pipeline runs on the merge request.
+1. A mainatiner from the parent project checks the pipeline result, and merge
+into a target branch if the latest pipeline has passed.
+
+Currently, those pipelines are created in a **forked** project, not in the
+parent project. This means you cannot completely trust the pipeline result,
+because, technically, external contributors can disguise their pipeline results
+by tweaking their GitLab Runner in the forked project.
+
+There are multiple reasons about why GitLab doesn't allow those pipelines to be
+created in the parent project, but one of the biggest reasons is security concern.
+External users could steal secret variables from the parent project by modifying
+.gitlab-ci.yml, which could be some sort of credentials. This should not happen.
+
+We're discussing a secure solution of running pipelines for merge requests
+that submitted from forked projects,
+see [the issue about the permission extension](https://gitlab.com/gitlab-org/gitlab-ce/issues/23902).

doc/ci/variables/README.md

@@ -41,7 +41,7 @@ strongly advised to use the new variables as we will remove the old ones in
 future GitLab releases.**
 
 | Variable                                  | GitLab | Runner | Description |
-|-------------------------------- |--------|--------|-------------|
+|-------------------------------------------|--------|--------|-------------|
 | **ARTIFACT_DOWNLOAD_ATTEMPTS**            | 8.15   | 1.9    | Number of attempts to download artifacts running a job |
 | **CI**                                    | all    | 0.4    | Mark that job is executed in CI environment |
 | **CI_COMMIT_REF_NAME**                    | 9.0    | all    | The branch or tag name for which project is built |
@@ -65,6 +65,17 @@ future GitLab releases.**
 | **CI_JOB_NAME**                           | 9.0    | 0.5    | The name of the job as defined in `.gitlab-ci.yml` |
 | **CI_JOB_STAGE**                          | 9.0    | 0.5    | The name of the stage as defined in `.gitlab-ci.yml` |
 | **CI_JOB_TOKEN**                          | 9.0    | 1.2    | Token used for authenticating with the [GitLab Container Registry][registry] and downloading [dependent repositories][dependent-repositories] |
+| **CI_MERGE_REQUEST_ID**                   | 11.6   | all    | The ID of the merge request if it's [pipelines for merge requests](../merge_request_pipelines/index.md) |
+| **CI_MERGE_REQUEST_IID**                  | 11.6   | all    | The IID of the merge request if it's [pipelines for merge requests](../merge_request_pipelines/index.md) |
+| **CI_MERGE_REQUEST_REF_PATH**             | 11.6   | all    | The ref path of the merge request if it's [pipelines for merge requests](../merge_request_pipelines/index.md). (e.g. `refs/merge-requests/1/head`) |
+| **CI_MERGE_REQUEST_PROJECT_ID**           | 11.6   | all    | The ID of the project of the merge request if it's [pipelines for merge requests](../merge_request_pipelines/index.md) |
+| **CI_MERGE_REQUEST_PROJECT_PATH**         | 11.6   | all    | The path of the project of the merge request if it's [pipelines for merge requests](../merge_request_pipelines/index.md) (e.g. `namespace/awesome-project`) |
+| **CI_MERGE_REQUEST_PROJECT_URL**          | 11.6   | all    | The URL of the project of the merge request if it's [pipelines for merge requests](../merge_request_pipelines/index.md) (e.g. `http://192.168.10.15:3000/namespace/awesome-project`) |
+| **CI_MERGE_REQUEST_TARGET_BRANCH_NAME**   | 11.6   | all    | The target branch name of the merge request if it's [pipelines for merge requests](../merge_request_pipelines/index.md) |
+| **CI_MERGE_REQUEST_SOURCE_PROJECT_ID**    | 11.6   | all    | The ID of the source project of the merge request if it's [pipelines for merge requests](../merge_request_pipelines/index.md) |
+| **CI_MERGE_REQUEST_SOURCE_PROJECT_PATH**  | 11.6   | all    | The path of the source project of the merge request if it's [pipelines for merge requests](../merge_request_pipelines/index.md) |
+| **CI_MERGE_REQUEST_SOURCE_PROJECT_URL**   | 11.6   | all    | The URL of the source project of the merge request if it's [pipelines for merge requests](../merge_request_pipelines/index.md) |
+| **CI_MERGE_REQUEST_SOURCE_BRANCH_NAME**   | 11.6   | all    | The source branch name of the merge request if it's [pipelines for merge requests](../merge_request_pipelines/index.md) |
 | **CI_NODE_INDEX**                         | 11.5   | all    | Index of the job in the job set. If the job is not parallelized, this variable is not set. |
 | **CI_NODE_TOTAL**                         | 11.5   | all    | Total number of instances of this job running in parallel. If the job is not parallelized, this variable is set to `1`. |
 | **CI_JOB_URL**                            | 11.1   | 0.5    | Job details URL |

doc/ci/yaml/README.md

@@ -342,8 +342,8 @@ In addition, `only` and `except` allow the use of special keywords:
 
 | **Value** |  **Description**  |
 | --------- |  ---------------- |
-| `branches`  | When a branch is pushed.  |
-| `tags`      | When a tag is pushed.  |
+| `branches`       | When a git reference of a pipeline is a branch.  |
+| `tags`           | When a git reference of a pipeline is a tag.  |
 | `api`            | When pipeline has been triggered by a second pipelines API (not triggers API).  |
 | `external`       | When using CI services other than GitLab. |
 | `pipelines`      | For multi-project triggers, created using the API with `CI_JOB_TOKEN`. |
@@ -351,6 +351,7 @@ In addition, `only` and `except` allow the use of special keywords:
 | `schedules`      | For [scheduled pipelines][schedules]. |
 | `triggers`       | For pipelines created using a trigger token. |
 | `web`            | For pipelines created using **Run pipeline** button in GitLab UI (under your project's **Pipelines**). |
+| `merge_requests` | When a merge request is created or updated (See [pipelines for merge requests](../merge_request_pipelines/index.md)). |
 
 In the example below, `job` will run only for refs that start with `issue-`,
 whereas all branches will be skipped:
@@ -391,6 +392,24 @@ job:
 The above example will run `job` for all branches on `gitlab-org/gitlab-ce`,
 except master.
 
+If a job does not have neither `only` nor `except` rule,
+`only: ['branches', 'tags']` is set by default.
+
+For example,
+
+```yaml
+job:
+  script: echo 'test'
+```
+
+is translated to
+
+```yaml
+job:
+  script: echo 'test'
+  only: ['branches', 'tags']
+```
+
 ## `only` and `except` (complex)
 
 > `refs` and `kubernetes` policies introduced in GitLab 10.0

doc/user/project/merge_requests/index.md

@@ -259,6 +259,16 @@ all your changes will be available to preview by anyone with the Review Apps lin
 
 [Read more about Review Apps.](../../../ci/review_apps/index.md)
 
+## Pipelines for merge requests
+
+When a developer updates a merge request, a pipeline should quickly report back
+its result to the developer, but often pipelines take long time to complete
+because general branch pipelines contain unnecessary jobs from the merge request standpoint.
+You can customize a specific pipeline structure for merge requests in order to
+speed the cycle up by running only important jobs.
+
+Learn more about [pipelines for merge requests](../../../ci/merge_request_pipelines/index.md).
+
 ## Pipeline status in merge requests
 
 If you've set up [GitLab CI/CD](../../../ci/README.md) in your project,