@@ -117,7 +117,7 @@ The following table lists available parameters for jobs:
| [`when`](#when) | When to run job. Also available: `when:manual` and `when:delayed`. |
| [`environment`](#environment) | Name of an environment to which the job deploys. Also available: `environment:name`, `environment:url`, `environment:on_stop`, `environment:auto_stop_in` and `environment:action`. |
| [`cache`](#cache) | List of files that should be cached between subsequent runs. Also available: `cache:paths`, `cache:key`, `cache:untracked`, and `cache:policy`. |
| [`artifacts`](#artifacts) | List of files and directories to attach to a job on success. Also available: `artifacts:paths`, `artifacts:exclude`, `artifacts:expose_as`, `artifacts:name`, `artifacts:untracked`, `artifacts:when`, `artifacts:expire_in`, `artifacts:reports`, `artifacts:reports:junit`, `artifacts:reports:cobertura`, and `artifacts:reports:terraform`.<br><br>In GitLab [Enterprise Edition](https://about.gitlab.com/pricing/), these are available: `artifacts:reports:codequality`, `artifacts:reports:sast`, `artifacts:reports:dependency_scanning`, `artifacts:reports:container_scanning`, `artifacts:reports:dast`, `artifacts:reports:license_scanning`, `artifacts:reports:license_management` (removed in GitLab 13.0),`artifacts:reports:performance` and `artifacts:reports:metrics`. |
| [`artifacts`](#artifacts) | List of files and directories to attach to a job on success. Also available: `artifacts:paths`, `artifacts:exclude`, `artifacts:expose_as`, `artifacts:name`, `artifacts:untracked`, `artifacts:when`, `artifacts:expire_in`, `artifacts:reports`, `artifacts:reports:junit`, `artifacts:reports:cobertura`, and `artifacts:reports:terraform`.<br><br>In GitLab [Enterprise Edition](https://about.gitlab.com/pricing/), these are available: `artifacts:reports:codequality`, `artifacts:reports:sast`, `artifacts:reports:dependency_scanning`, `artifacts:reports:container_scanning`, `artifacts:reports:dast`, `artifacts:reports:license_scanning`, `artifacts:reports:license_management` (removed in GitLab 13.0),`artifacts:reports:performance` and `artifacts:reports:metrics`. |
| [`dependencies`](#dependencies) | Restrict which artifacts are passed to a specific job by providing a list of jobs to fetch artifacts from. |
| [`coverage`](#coverage) | Code coverage settings for a given job. |
| [`retry`](#retry) | When and how many times a job can be auto-retried in case of a failure. |
...
...
@@ -3148,7 +3148,7 @@ These are the available report types:
1. After configuring the Runner, add a new job to `.gitlab-ci.yml` that generates
the expected report.
1. Define the `performance` job according to your version of GitLab:
- For GitLab 12.4 and later - [include](../../../ci/yaml/README.md#includetemplate) the
[`Browser-Performance.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Verify/Browser-Performance.gitlab-ci.yml) provided as a part of your GitLab installation.
- For GitLab versions earlier than 12.4 - Copy and use the job as defined in the
For versions before 12.4, see the information for [older GitLab versions](#gitlab-versions-123-and-older).
If you are using a Kubernetes cluster, use [`template: Jobs/Browser-Performance-Testing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Browser-Performance-Testing.gitlab-ci.yml)
instead.
The above example creates a `performance` job in your CI/CD pipeline and runs
sitespeed.io against the webpage you defined in `URL` to gather key metrics.
The [GitLab plugin for sitespeed.io](https://gitlab.com/gitlab-org/gl-performance)
is downloaded to save the report as a [Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsperformance-premium)
that you can later download and analyze. Due to implementation limitations, we always
take the latest Performance artifact available.
The full HTML sitespeed.io report is saved as an artifact, and if
[GitLab Pages](../pages/index.md) is enabled, it can be viewed directly in your browser.
The example uses a CI/CD template that is included in all GitLab installations since
12.4, but it will not work with Kubernetes clusters. If you are using GitLab 12.3
or older, you must [add the configuration manually](#gitlab-versions-123-and-older)
The template uses the [GitLab plugin for sitespeed.io](https://gitlab.com/gitlab-org/gl-performance),
and it saves the full HTML sitespeed.io report as a [Browser Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsperformance-premium)
that you can later download and analyze. This implementation always takes the latest
Browser Performance artifact available. If [GitLab Pages](../pages/index.md) is enabled,
you can view the report directly in your browser.
You can also customize the jobs with environment variables:
-`SITESPEED_IMAGE`: Configure the Docker image to use for the job (default `sitespeedio/sitespeed.io`), but not the image version.
-`SITESPEED_VERSION`: Configure the version of the Docker image to use for the job (default `13.3.0`).
-`SITESPEED_OPTIONS`: Configure any additional sitespeed.io options as required (default `nil`). Refer to the [sitespeed.io documentation](https://www.sitespeed.io/documentation/sitespeed.io/configuration/) for more details.
You can also customize options by setting the `SITESPEED_OPTIONS` variable.
For example, you can override the number of runs sitespeed.io
makes on the given URL:
makes on the given URL, and change the version:
```yaml
include:
...
...
@@ -114,18 +105,11 @@ include:
performance:
variables:
URL:https://example.com
URL:https://www.sitespeed.io/
SITESPEED_VERSION:13.2.0
SITESPEED_OPTIONS:-n 5
```
For further customization options for sitespeed.io, including the ability to provide a
Key metrics are automatically extracted and shown in the merge request widget.
### Configuring degradation threshold
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27599) in GitLab 13.0.
...
...
@@ -152,15 +136,12 @@ The above CI YAML configuration is great for testing against static environments
be extended for dynamic environments, but a few extra steps are required:
1. The `performance` job should run after the dynamic environment has started.
1. In the `review` job, persist the hostname and upload it as an artifact so
it's available to the `performance` job. The same can be done for static
environments like staging and production to unify the code path. You can save it
as an artifact with `echo $CI_ENVIRONMENT_URL > environment_url.txt`
in your job's `script`.
1. In the `performance` job, read the previous artifact into an environment
variable. In this case, use `$URL` because the sitespeed.io command
uses it for the URL parameter. Because Review App URLs are dynamic, define
the `URL` variable through `before_script` instead of `variables`.
1. In the `review` job:
1. Generate a URL list file with the dynamic URL.
1. Save the file as an artifact, for example with `echo $CI_ENVIRONMENT_URL > environment_url.txt`
in your job's `script`.
1. Pass the list as the URL environment variable (which can be a URL or a file containing URLs)
to the `performance` job.
1. You can now run the sitespeed.io container against the desired hostname and
paths.
...
...
@@ -193,20 +174,21 @@ review:
performance:
dependencies:
-review
before_script:
-export URL=$(cat environment_url.txt)
variables:
URL:environment_url.txt
```
### Previous job definitions
### GitLab versions 12.3 and older
CAUTION: **Caution:**
Before GitLab 11.5, the Performance job and artifact had to be named specifically
to automatically extract report data and show it in the merge request widget.
While these old job definitions are still maintained, they have been deprecated
and may be removed in next major release, GitLab 12.0.
GitLab recommends you update your current `.gitlab-ci.yml` configuration to reflect that change.
Browser Performance Testing has gone through several changes since it's introduction.
In this section we'll detail these changes and how you can run the test based on your
GitLab version:
For GitLab 11.4 and earlier, the job should look like:
- In GitLab 12.4 [a job template was made available](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Verify/Browser-Performance.gitlab-ci.yml).
- In 13.2 the feature was renamed from `Performance` to `Browser Performance` with
additional template variables. The job name in the template is still `performance`
for compatibility reasons, but may be renamed to match in a future iteration.
- For 11.5 to 12.3 no template is available and the job has to be defined manually as follows:
