Merge branch 'selhorn-moved-yaml-specific-features-to-new-file' into 'master'

Moved yaml-specific features to new page

See merge request gitlab-org/gitlab!72646

doc/ci/caching/index.md

@@ -134,7 +134,7 @@ job:
 ## Inherit global configuration, but override specific settings per job
 
 You can override cache settings without overwriting the global cache by using
-[anchors](../yaml/index.md#anchors). For example, if you want to override the
+[anchors](../yaml/yaml_specific_features.md#anchors). For example, if you want to override the
 `policy` for one job:
 
 ```yaml

doc/ci/jobs/index.md

@@ -139,6 +139,32 @@ In [GitLab 13.8 and earlier](https://gitlab.com/gitlab-org/gitlab/-/merge_reques
 the regular expression is `\d+[\s:\/\\]+\d+\s*`. [Feature flag](../../user/feature_flags.md)
 removed in [GitLab 13.11](https://gitlab.com/gitlab-org/gitlab/-/issues/322080).
 
+## Hide jobs
+
+To temporarily disable a job without deleting it from the configuration
+file:
+
+- Comment out the job's configuration:
+
+  ```yaml
+  # hidden_job:
+  #   script:
+  #     - run test
+  ```
+
+- Start the job name with a dot (`.`) and it is not processed by GitLab CI/CD:
+
+```yaml
+.hidden_job:
+  script:
+    - run test
+```
+
+You can use hidden jobs that start with `.` as templates for reusable configuration with:
+
+- The [`extends` keyword](../yaml/index.md#extends).
+- [YAML anchors](../yaml/yaml_specific_features.md#anchors).
+
 ## Specifying variables when running manual jobs
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30485) in GitLab 12.2.

doc/ci/jobs/job_control.md

@@ -294,7 +294,7 @@ You can use the `$` character for both variables and paths. For example, if the
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322992) in GitLab 14.3.
 
-Use [`!reference` tags](../yaml/index.md#reference-tags) to reuse rules in different
+Use [`!reference` tags](../yaml/yaml_specific_features.md#reference-tags) to reuse rules in different
 jobs. You can combine `!reference` rules with regular job-defined rules:
 
 ```yaml

doc/ci/pipeline_editor/index.md

@@ -83,7 +83,7 @@ where:
 - Configuration imported with [`include`](../yaml/index.md#include) is copied into the view.
 - Jobs that use [`extends`](../yaml/index.md#extends) display with the
   [extended configuration merged into the job](../yaml/index.md#merge-details).
-- YAML anchors are [replaced with the linked configuration](../yaml/index.md#anchors).
+- YAML anchors are [replaced with the linked configuration](../yaml/yaml_specific_features.md#anchors).
 
 ## Commit changes to CI configuration
 

doc/ci/pipelines/index.md

@@ -229,6 +229,15 @@ This functionality is only available:
 - For users with at least the Developer role.
 - If the stage contains [manual actions](#add-manual-interaction-to-your-pipeline).
 
+### Skip a pipeline
+
+To push a commit without triggering a pipeline, add `[ci skip]` or `[skip ci]`, using any
+capitalization, to your commit message.
+
+Alternatively, if you are using Git 2.10 or later, use the `ci.skip` [Git push option](../../user/project/push_options.md#push-options-for-gitlab-cicd).
+The `ci.skip` push option does not skip merge request
+pipelines.
+
 ### Delete a pipeline
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24851) in GitLab 12.7.

doc/ci/yaml/gitlab_ci_yaml.md

@@ -24,7 +24,7 @@ In the `.gitlab-ci.yml` file, you can define:
 
 The scripts are grouped into **jobs**, and jobs run as part of a larger
 **pipeline**. You can group multiple independent jobs into **stages** that run in a defined order.
-The CI/CD configuration needs at least one job that is [not hidden](index.md#hide-jobs).
+The CI/CD configuration needs at least one job that is [not hidden](../jobs/index.md#hide-jobs).
 
 You should organize your jobs in a sequence that suits your application and is in accordance with
 the tests you wish to perform. To [visualize](../pipeline_editor/index.md#visualize-ci-configuration) the process, imagine

doc/ci/yaml/yaml_specific_features.md

+---
+stage: Verify
+group: Pipeline Authoring
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+type: reference
+---
+
+# YAML-specific features
+
+In your `.gitlab-ci.yml` file, you can use YAML-specific features like anchors (`&`), aliases (`*`),
+and map merging (`<<`). Use these features to reduce the complexity
+of the code in the `.gitlab-ci.yml` file.
+
+Read more about the various [YAML features](https://learnxinyminutes.com/docs/yaml/).
+
+In most cases, the [`extends` keyword](index.md#extends) is more user friendly and you should
+use it when possible.
+
+You can use YAML anchors to merge YAML arrays.
+
+## Anchors
+
+YAML has a feature called 'anchors' that you can use to duplicate
+content across your document.
+
+Use anchors to duplicate or inherit properties. Use anchors with [hidden jobs](../jobs/index.md#hide-jobs)
+to provide templates for your jobs. When there are duplicate keys, GitLab
+performs a reverse deep merge based on the keys.
+
+You can't use YAML anchors across multiple files when using the [`include`](index.md#include)
+keyword. Anchors are only valid in the file they were defined in. To reuse configuration
+from different YAML files, use [`!reference` tags](#reference-tags) or the
+[`extends` keyword](index.md#extends).
+
+The following example uses anchors and map merging. It creates two jobs,
+`test1` and `test2`, that inherit the `.job_template` configuration, each
+with their own custom `script` defined:
+
+```yaml
+.job_template: &job_configuration  # Hidden yaml configuration that defines an anchor named 'job_configuration'
+  image: ruby:2.6
+  services:
+    - postgres
+    - redis
+
+test1:
+  <<: *job_configuration           # Merge the contents of the 'job_configuration' alias
+  script:
+    - test1 project
+
+test2:
+  <<: *job_configuration           # Merge the contents of the 'job_configuration' alias
+  script:
+    - test2 project
+```
+
+`&` sets up the name of the anchor (`job_configuration`), `<<` means "merge the
+given hash into the current one," and `*` includes the named anchor
+(`job_configuration` again). The expanded version of this example is:
+
+```yaml
+.job_template:
+  image: ruby:2.6
+  services:
+    - postgres
+    - redis
+
+test1:
+  image: ruby:2.6
+  services:
+    - postgres
+    - redis
+  script:
+    - test1 project
+
+test2:
+  image: ruby:2.6
+  services:
+    - postgres
+    - redis
+  script:
+    - test2 project
+```
+
+You can use anchors to define two sets of services. For example, `test:postgres`
+and `test:mysql` share the `script` defined in `.job_template`, but use different
+`services`, defined in `.postgres_services` and `.mysql_services`:
+
+```yaml
+.job_template: &job_configuration
+  script:
+    - test project
+  tags:
+    - dev
+
+.postgres_services:
+  services: &postgres_configuration
+    - postgres
+    - ruby
+
+.mysql_services:
+  services: &mysql_configuration
+    - mysql
+    - ruby
+
+test:postgres:
+  <<: *job_configuration
+  services: *postgres_configuration
+  tags:
+    - postgres
+
+test:mysql:
+  <<: *job_configuration
+  services: *mysql_configuration
+```
+
+The expanded version is:
+
+```yaml
+.job_template:
+  script:
+    - test project
+  tags:
+    - dev
+
+.postgres_services:
+  services:
+    - postgres
+    - ruby
+
+.mysql_services:
+  services:
+    - mysql
+    - ruby
+
+test:postgres:
+  script:
+    - test project
+  services:
+    - postgres
+    - ruby
+  tags:
+    - postgres
+
+test:mysql:
+  script:
+    - test project
+  services:
+    - mysql
+    - ruby
+  tags:
+    - dev
+```
+
+You can see that the hidden jobs are conveniently used as templates, and
+`tags: [postgres]` overwrites `tags: [dev]`.
+
+### YAML anchors for scripts
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23005) in GitLab 12.5.
+
+You can use [YAML anchors](#anchors) with [script](index.md#script), [`before_script`](index.md#before_script),
+and [`after_script`](index.md#after_script) to use predefined commands in multiple jobs:
+
+```yaml
+.some-script-before: &some-script-before
+  - echo "Execute this script first"
+
+.some-script: &some-script
+  - echo "Execute this script second"
+  - echo "Execute this script too"
+
+.some-script-after: &some-script-after
+  - echo "Execute this script last"
+
+job1:
+  before_script:
+    - *some-script-before
+  script:
+    - *some-script
+    - echo "Execute something, for this job only"
+  after_script:
+    - *some-script-after
+
+job2:
+  script:
+    - *some-script-before
+    - *some-script
+    - echo "Execute something else, for this job only"
+    - *some-script-after
+```
+
+### YAML anchors for variables
+
+Use [YAML anchors](#anchors) with `variables` to repeat assignment
+of variables across multiple jobs. You can also use YAML anchors when a job
+requires a specific `variables` block that would otherwise override the global variables.
+
+The following example shows how override the `GIT_STRATEGY` variable without affecting
+the use of the `SAMPLE_VARIABLE` variable:
+
+```yaml
+# global variables
+variables: &global-variables
+  SAMPLE_VARIABLE: sample_variable_value
+  ANOTHER_SAMPLE_VARIABLE: another_sample_variable_value
+
+# a job that must set the GIT_STRATEGY variable, yet depend on global variables
+job_no_git_strategy:
+  stage: cleanup
+  variables:
+    <<: *global-variables
+    GIT_STRATEGY: none
+  script: echo $SAMPLE_VARIABLE
+```
+
+## `!reference` tags
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/266173) in GitLab 13.9.
+> - `rules` keyword support [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322992) in GitLab 14.3.
+
+Use the `!reference` custom YAML tag to select keyword configuration from other job
+sections and reuse it in the current section. Unlike [YAML anchors](#anchors), you can
+use `!reference` tags to reuse configuration from [included](index.md#include) configuration
+files as well.
+
+In the following example, a `script` and an `after_script` from two different locations are
+reused in the `test` job:
+
+- `setup.yml`:
+
+  ```yaml
+  .setup:
+    script:
+      - echo creating environment
+  ```
+
+- `.gitlab-ci.yml`:
+
+  ```yaml
+  include:
+    - local: setup.yml
+
+  .teardown:
+    after_script:
+      - echo deleting environment
+
+  test:
+    script:
+      - !reference [.setup, script]
+      - echo running my own command
+    after_script:
+      - !reference [.teardown, after_script]
+  ```
+
+In the following example, `test-vars-1` reuses all the variables in `.vars`, while `test-vars-2`
+selects a specific variable and reuses it as a new `MY_VAR` variable.
+
+```yaml
+.vars:
+  variables:
+    URL: "http://my-url.internal"
+    IMPORTANT_VAR: "the details"
+
+test-vars-1:
+  variables: !reference [.vars, variables]
+  script:
+    - printenv
+
+test-vars-2:
+  variables:
+    MY_VAR: !reference [.vars, variables, IMPORTANT_VAR]
+  script:
+    - printenv
+```
+
+You can't reuse a section that already includes a `!reference` tag. Only one level
+of nesting is supported.

doc/development/pipelines.md

@@ -664,7 +664,7 @@ then included in individual jobs via [`extends`](../ci/yaml/index.md#extends).
 The `rules` definitions are composed of `if:` conditions and `changes:` patterns,
 which are also defined in
 [`rules.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rules.gitlab-ci.yml)
-and included in `rules` definitions via [YAML anchors](../ci/yaml/index.md#anchors)
+and included in `rules` definitions via [YAML anchors](../ci/yaml/yaml_specific_features.md#anchors)
 
 #### `if:` conditions
 

doc/user/application_security/coverage_fuzzing/index.md

@@ -70,7 +70,7 @@ my_fuzz_target:
     - ./gitlab-cov-fuzz run --regression=$REGRESSION -- <your fuzz target>
 ```
 
-The included template makes available the [hidden job](../../../ci/yaml/index.md#hide-jobs)
+The included template makes available the [hidden job](../../../ci/jobs/index.md#hide-jobs)
 `.fuzz_base`, which you must [extend](../../../ci/yaml/index.md#extends) for each of your fuzz
 targets. Each fuzz target **must** have a separate job. For example, the
 [go-fuzzing-example project](https://gitlab.com/gitlab-org/security-products/demos/go-fuzzing-example)

doc/user/project/merge_requests/merge_when_pipeline_succeeds.md

@@ -101,7 +101,7 @@ for details on avoiding two pipelines for a single merge request.
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211482) in GitLab 13.1.
 
-When the **Pipelines must succeed** checkbox is checked, [skipped pipelines](../../../ci/yaml/index.md#skip-pipeline) prevent
+When the **Pipelines must succeed** checkbox is checked, [skipped pipelines](../../../ci/pipelines/index.md#skip-a-pipeline) prevent
 merge requests from being merged. To change this behavior:
 
 1. Navigate to your project's **Settings > General** page.

doc/user/project/repository/index.md

@@ -41,7 +41,7 @@ to a branch in the repository. When you use the command line, you can commit mul
   If the project is configured with [GitLab CI/CD](../../../ci/index.md),
   you trigger a pipeline per push, not per commit.
   - **Skip pipelines:**
-  Add the [`ci skip`](../../../ci/yaml/index.md#skip-pipeline) keyword to
+  Add the [`ci skip`](../../../ci/pipelines/index.md#skip-a-pipeline) keyword to
   your commit message to make GitLab CI/CD skip the pipeline.
   - **Cross-link issues and merge requests:**
   Use [cross-linking](../issues/crosslinking_issues.md#from-commit-messages)