Merge branch 'doc-rules' into 'master'

Improve the pipelines documentation with the latest rules

See merge request gitlab-org/gitlab!24485

.gitlab-ci.yml

@@ -3,7 +3,6 @@ image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.6.5-golang-1.1
 stages:
   - sync
   - prepare
-  - quick-test
   - test
   - post-test
   - review-prepare

doc/development/pipelines.md

@@ -19,9 +19,6 @@ The current stages are:
   <https://gitlab.com/gitlab-org/gitlab-foss>.
 - `prepare`: This stage includes jobs that prepare artifacts that are needed by
   jobs in subsequent stages.
-- `quick-test`: This stage includes test jobs that should run first and fail the
-  pipeline early (currently used to run Geo tests when the branch name starts
-  with `geo-`, `geo/`, or ends with `-geo`).
 - `test`: This stage includes most of the tests, DB/migration jobs, and static analysis jobs.
 - `post-test`: This stage includes jobs that build reports or gather data from
   the `test` stage's jobs (e.g. coverage, Knapsack metadata etc.).
@@ -138,54 +135,109 @@ duplicating the `if` conditions and `changes` patterns lists since they cannot b
 **If you update an `if` condition or `changes`
 patterns list, make sure to mass-update those across all the CI config files (i.e. `.gitlab/ci/*.yml`).**
 
-### Canonical commits only
+### Canonical/security namespace merge requests only
 
-This condition limits jobs creation to commits under the `gitlab-org/` top-level group
-on GitLab.com only. This is similar to the `.only:variables-canonical-dot-com` CI definition:
+This condition limits jobs creation to merge requests under the `gitlab-org/` top-level group
+on GitLab.com only (i.e. this won't run for `master`, stable or auto-deploy branches).
+This is similar to the `.only:variables-canonical-dot-com` + `only:refs: [merge_requests]`
+CI definitions.
 
-```yaml
-.if-canonical-gitlab: &if-canonical-gitlab
-  if: '$CI_SERVER_HOST == "gitlab.com" && $CI_PROJECT_NAMESPACE =~ /^gitlab-org($|\/)/'
-```
+The definition for `if-canonical-dot-com-gitlab-org-groups-merge-request` can be
+seen in <https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/docs.gitlab-ci.yml>.
 
-### Canonical merge requests only
+### Canonical/security namespace tags only
 
-Same as the "Canonical commits only" condition above but further limits jobs creation
-to merge requests only (i.e. this won't run for `master`, stable or auto-deploy branches).
-This is similar to the `.only:variables-canonical-dot-com` + `.except:refs-master-tags-stable-deploy`
-CI definitions:
+This condition limits jobs creation to tags under the `gitlab-org/` top-level group
+on GitLab.com only.
+This is similar to the `.only:variables-canonical-dot-com` + `only:refs: [tags]` CI definition:
 
-```yaml
-.if-canonical-gitlab-merge-request: &if-canonical-gitlab-merge-request
-  if: '$CI_SERVER_HOST == "gitlab.com" && $CI_PROJECT_NAMESPACE =~ /^gitlab-org($|\/)/ && $CI_MERGE_REQUEST_IID'
-```
+The definition for `if-canonical-dot-com-gitlab-org-groups-tag` can be seen in
+<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/cng.gitlab-ci.yml>.
+
+### Canonical namespace `master` only
+
+This condition limits jobs creation to `master` pipelines for the `gitlab-org` top-level group
+on GitLab.com only.
+This is similar to the `.only:variables-canonical-dot-com` + `only:refs: [master]` CI definition:
+
+The definition for `if-canonical-dot-com-gitlab-org-group-master-refs` can be
+seen in <https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/pages.gitlab-ci.yml>.
+
+### Canonical namespace schedules only
+
+This condition limits jobs creation to scheduled pipelines for the `gitlab-org` top-level group
+on GitLab.com only.
+This is similar to the `.only:variables-canonical-dot-com` + `only:refs: [schedules]` CI definition:
+
+The definition for `if-canonical-dot-com-gitlab-org-group-schedule` can be seen
+in <https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/qa.gitlab-ci.yml>.
+
+### Not canonical/security namespace
+
+This condition matches if the project isn't in the canonical/security namespace.
+Useful to **not** create a job if the project is a fork, or in other words, when
+a job should only run in the canonical projects.
+
+The definition for `if-not-canonical-namespace` can be seen in
+<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/frontend.gitlab-ci.yml>.
+
+### Not EE
+
+This condition matches if the project isn't EE. Useful to **not** create a job if
+the project is GitLab, or in other words, when a job should only run in the GitLab
+FOSS project.
+
+The definition for `if-not-ee` can be seen in
+<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/frontend.gitlab-ci.yml>.
+
+### Default refs only
+
+This condition is the equivalent of `.default-only`.
+
+The definition for `if-default-refs` can be seen in
+<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/frontend.gitlab-ci.yml>.
+
+### `master` refs only
+
+This condition is the equivalent of `only:refs: [master]`.
+
+The definition for `if-master-refs` can be seen in
+<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/frontend.gitlab-ci.yml>.
 
 ### Code changes patterns
 
 Similar patterns as for `.only:changes-code`:
 
-```yaml
-.code-patterns: &code-patterns
-  - ...
-```
+The definition for `code-patterns` can be seen in
+<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/qa.gitlab-ci.yml>.
 
 ### QA changes patterns
 
 Similar patterns as for `.only:changes-qa`:
 
-```yaml
-.qa-patterns: &qa-patterns
-  - ...
-```
+The definition for `qa-patterns` can be seen in
+<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/qa.gitlab-ci.yml>.
+
+### Docs changes patterns
+
+Similar patterns as for `.only:changes-docs`:
+
+The definition for `docs-patterns` can be seen in
+<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/docs.gitlab-ci.yml>.
 
 ### Code and QA changes patterns
 
 Similar patterns as for `.only:changes-code-qa`:
 
-```yaml
-.code-qa-patterns: &code-qa-patterns
-  - ...
-```
+The definition for `code-qa-patterns` can be seen in
+<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/review.gitlab-ci.yml>.
+
+### Code, backstage and QA changes patterns
+
+Similar patterns as for `.only:changes-code-backstage-qa`:
+
+The definition for `code-backstage-qa-patterns` can be seen in
+<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/frontend.gitlab-ci.yml>.
 
 ## Directed acyclic graph
 
@@ -195,21 +247,21 @@ execute jobs out of order for the following jobs:
 ```mermaid
 graph RL;
   A[setup-test-env];
-  B["gitlab:assets:compile pull-push-cache<br/>(master only)"];
-  C[gitlab:assets:compile pull-cache];
+  B["gitlab:assets:compile pull-push-cache<br/>(canonical master only)"];
+  C["gitlab:assets:compile pull-cache<br/>(canonical default refs only)"];
   D["cache gems<br/>(master and tags only)"];
   E[review-build-cng];
   F[build-qa-image];
   G[review-deploy];
   G2["schedule:review-deploy<br/>(master only)"];
-  H[karma];
-  I[jest];
+  I["karma, jest, webpack-dev-server, static-analysis"];
+  I2["karma-foss, jest-foss<br/>(EE default refs only)"];
   J["compile-assets pull-push-cache<br/>(master only)"];
+  J2["compile-assets pull-push-cache foss<br/>(EE master only)"];
   K[compile-assets pull-cache];
-  L[webpack-dev-server];
+  K2["compile-assets pull-cache foss<br/>(EE default refs only)"];
   M[coverage];
-  N[pages];
-  O[static-analysis];
+  N["pages (master only)"];
   Q[package-and-qa];
   S["RSpec<br/>(e.g. rspec unit pg9)"]
   T[retrieve-tests-metadata];
@@ -220,58 +272,55 @@ subgraph "`prepare` stage"
     C
     F
     K
+    K2
     J
+    J2
     T
     end
 
 subgraph "`test` stage"
-    D --> |needs| A;
-    H -.-> |needs and depends on| A;
-    H -.-> |needs and depends on| K;
+    D -.-> |needs| A;
     I -.-> |needs and depends on| A;
     I -.-> |needs and depends on| K;
+    I2 -.-> |needs and depends on| A;
+    I2 -.-> |needs and depends on| K;
     L -.-> |needs and depends on| A;
-    L -.-> |needs and depends on| K;
-    O -.-> |needs and depends on| A;
-    O -.-> |needs and depends on| K;
     S -.-> |needs and depends on| A;
     S -.-> |needs and depends on| K;
     S -.-> |needs and depends on| T;
-    downtime_check --> |needs and depends on| A;
-    db:* --> |needs| A;
-    gitlab:setup --> |needs| A;
-    downtime_check --> |needs and depends on| A;
-    graphql-docs-verify --> |needs| A;
+    L["db:*, gitlab:setup, graphql-docs-verify, downtime_check"] -.-> |needs| A;
+    end
+
+subgraph "`post-test` stage"
+    M --> |happens after| S
     end
 
 subgraph "`review-prepare` stage"
-    E --> |needs| C;
-    X["schedule:review-build-cng<br/>(master schedule only)"] --> |needs| C;
+    E -.-> |needs| C;
+    E2["schedule:review-build-cng<br/>(master schedule only)"] -.-> |needs| C;
     end
 
 subgraph "`review` stage"
-    G
-    G2
+    G --> |happens after| E
+    G2 --> |happens after| E2
     end
 
 subgraph "`qa` stage"
-    Q --> |needs| C;
-    Q --> |needs| F;
-    review-qa-smoke -.-> |needs and depends on| G;
-    review-qa-all -.-> |needs and depends on| G;
-    review-performance -.-> |needs and depends on| G;
-    X2["schedule:review-performance<br/>(master only)"] -.-> |needs and depends on| G2;
-    dast -.-> |needs and depends on| G;
+    Q -.-> |needs| C;
+    Q -.-> |needs| F;
+    QA1["review-qa-smoke, review-qa-all, review-performance, dast"] -.-> |needs and depends on| G;
+    QA2["schedule:review-performance<br/>(master only)"] -.-> |needs and depends on| G2;
     end
 
-subgraph "`post-test` stage"
-    M
-    end
+subgraph "`post-qa` stage"
+  PQA1["parallel-spec-reports"] -.-> |depends on `review-qa-all`| QA1;
+  end
 
 subgraph "`pages` stage"
     N -.-> |depends on| C;
-    N -.-> |depends on| H;
+    N -.-> |depends on karma| I;
     N -.-> |depends on| M;
+    N --> |happens after| PQA1
     end
 ```