Skip to content

G
gitlab-ce
Commit 858d093b authored by Olivier Gonzalez's avatar Olivier Gonzalez Committed by Achilleas Pipinellis
Browse files
Options

Document new report types

parent 2356f9ab
Hide whitespace changes
Inline Side-by-side
Showing with 318 additions and 64 deletions
doc/ci/examples/browser_performance.md
View file @ 858d093b
# Browser Performance Testing with the Sitespeed.io container
CAUTION: **Caution:**
The job definition shown below is supported on GitLab 11.5 and later versions.
It also requires the GitLab Runner 11.5 or later.
For earlier versions, use the [previous job definitions](#previous-job-definitions).
This example shows how to run the
[Sitespeed.io container](https://hub.docker.com/r/sitespeedio/sitespeed.io/) on
your code by using GitLab CI/CD and [Sitespeed.io](https://www.sitespeed.io)
using Docker-in-Docker.
First, you need a GitLab Runner with the
First, you need GitLab Runner with
[docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-executor).
Once you set up the Runner, add a new job to `.gitlab-ci.yml`, called
`performance`:
Once you set up the Runner, add a new job to `.gitlab-ci.yml` that
generates the expected report:
```yaml
performance:
......@@ -26,19 +32,22 @@ performance:
- mv sitespeed-results/data/performance.json performance.json
artifacts:
paths:
- performance.json
- sitespeed-results/
- sitespeed-results/
reports:
performance: performance.json
```
The above example will:
The above example will create a `performance` job in your CI/CD pipeline and will run
Sitespeed.io against the webpage you defined in `URL` to gather key metrics.
The [GitLab plugin](https://gitlab.com/gitlab-org/gl-performance) for
Sitespeed.io is downloaded in order to save the report as a
[Performance report artifact](https://docs.gitlab.com/ee//ci/yaml/README.html#artifactsreportsperformance)
that you can later download and analyze.
Due to implementation limitations we always take the latest Performance artifact available.
1. Create a `performance` job in your CI/CD pipeline and will run
Sitespeed.io against the webpage you defined in `URL`.
1. The [GitLab plugin](https://gitlab.com/gitlab-org/gl-performance) for
Sitespeed.io is downloaded in order to export key metrics to JSON. The full
HTML Sitespeed.io report will also be saved as an artifact, and if you have
[GitLab Pages](../../user/project/pages/index.md) enabled, it can be viewed
directly in your browser.
The full HTML Sitespeed.io report will also be saved as an artifact, and if you have
[GitLab Pages](../../user/project/pages/index.md) enabled, it can be viewed
directly in your browser.
For further customization options of Sitespeed.io, including the ability to
provide a list of URLs to test, please consult
......@@ -46,8 +55,8 @@ provide a list of URLs to test, please consult
TIP: **Tip:**
For [GitLab Premium](https://about.gitlab.com/pricing/) users, key metrics are automatically
extracted and shown right in the merge request widget. Learn more about
[Browser Performance Testing](https://docs.gitlab.com/ee/user/project/merge_requests/browser_performance_testing.html).
extracted and shown right in the merge request widget.
[Learn more on Browser Performance Testing in merge requests](https://docs.gitlab.com/ee//user/project/merge_requests/browser_performance_testing.html).
## Performance testing on Review Apps
......@@ -106,8 +115,40 @@ performance:
- mv sitespeed-results/data/performance.json performance.json
artifacts:
paths:
- performance.json
- sitespeed-results/
reports:
performance: performance.json
```
A complete example can be found in our [Auto DevOps CI YML](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml).
## Previous job definitions
CAUTION: **Caution:**
Before GitLab 11.5, Performance job and artifact had to be named specifically
to automatically extract report data and show it in the merge request widget.
While these old job definitions are still maintained they have been deprecated
and may be removed in next major release, GitLab 12.0.
You are advised to update your current `.gitlab-ci.yml` configuration to reflect that change.
For GitLab 11.4 and earlier, the job should look like:
```yaml
performance:
stage: performance
image: docker:git
variables:
URL: https://example.com
services:
- docker:stable-dind
script:
- mkdir gitlab-exporter
- wget -O ./gitlab-exporter/index.js https://gitlab.com/gitlab-org/gl-performance/raw/master/index.js
- mkdir sitespeed-results
- docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:6.3.1 --plugins.add ./gitlab-exporter --outputFolder sitespeed-results $URL
- mv sitespeed-results/data/performance.json performance.json
artifacts:
paths:
- performance.json
- sitespeed-results/
```
\ No newline at end of file
doc/ci/examples/code_quality.md
View file @ 858d093b
# Analyze your project's Code Quality
CAUTION: **Caution:**
The job definition shown below is supported on GitLab 11.5 and later versions.
It also requires the GitLab Runner 11.5 or later.
For earlier versions, use the [previous job definitions](#previous-job-definitions).
This example shows how to run Code Quality on your code by using GitLab CI/CD
and Docker.
First, you need GitLab Runner with [docker-in-docker executor][dind].
First, you need GitLab Runner with
[docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-executor).
Once you set up the Runner, add a new job to `.gitlab-ci.yml`, called `code_quality`:
Once you set up the Runner, add a new job to `.gitlab-ci.yml` that
generates the expected report:
```yaml
code_quality:
......@@ -23,27 +30,72 @@ code_quality:
--volume /var/run/docker.sock:/var/run/docker.sock
"registry.gitlab.com/gitlab-org/security-products/codequality:$SP_VERSION" /code
artifacts:
paths: [gl-code-quality-report.json]
reports:
codequality: gl-code-quality-report.json
```
The above example will create a `code_quality` job in your CI/CD pipeline which
will scan your source code for code quality issues. The report will be saved
as an artifact that you can later download and analyze.
will scan your source code for code quality issues. The report will be saved as a
[Code Quality report artifact](../../ci/yaml/README.md#artifactsreportscodequality)
that you can later download and analyze.
Due to implementation limitations we always take the latest Code Quality artifact available.
TIP: **Tip:**
Starting with [GitLab Starter][ee] 9.3, this information will
be automatically extracted and shown right in the merge request widget. To do
so, the CI/CD job must be named `code_quality` and the artifact path must be
`gl-code-quality-report.json`.
For [GitLab Starter][ee] users, this information will be automatically
extracted and shown right in the merge request widget.
[Learn more on Code Quality in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality.html).
## Previous job definitions
CAUTION: **Caution:**
Code Quality was previously using `codeclimate` and `codequality` for job name and
`codeclimate.json` for the artifact name. While these old names
are still maintained they have been deprecated with GitLab 11.0 and may be removed
in next major release, GitLab 12.0. You are advised to update your current `.gitlab-ci.yml`
configuration to reflect that change.
Before GitLab 11.5, Code Quality job and artifact had to be named specifically
to automatically extract report data and show it in the merge request widget.
While these old job definitions are still maintained they have been deprecated
and may be removed in next major release, GitLab 12.0.
You are advised to update your current `.gitlab-ci.yml` configuration to reflect that change.
For GitLab 11.4 and earlier, the job should look like:
```yaml
code_quality:
image: docker:stable
variables:
DOCKER_DRIVER: overlay2
allow_failure: true
services:
- docker:stable-dind
script:
- export SP_VERSION=$(echo "$CI_SERVER_VERSION" | sed 's/^\([0-9]*\)\.\([0-9]*\).*/\1-\2-stable/')
- docker run
--env SOURCE_CODE="$PWD"
--volume "$PWD":/code
--volume /var/run/docker.sock:/var/run/docker.sock
"registry.gitlab.com/gitlab-org/security-products/codequality:$SP_VERSION" /code
artifacts:
paths: [gl-code-quality-report.json]
```
Alternatively the job name could be `codeclimate` or `codequality`
and the artifact name could be `codeclimate.json`.
These names have been deprecated with GitLab 11.0
and may be removed in next major release, GitLab 12.0.
For GitLab 10.3 and earlier, the job should look like:
```yaml
codequality:
image: docker:latest
variables:
DOCKER_DRIVER: overlay
services:
- docker:dind
script:
- docker pull codeclimate/codeclimate:0.69.0
- docker run --env CODECLIMATE_CODE="$PWD" --volume "$PWD":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate:0.69.0 init
- docker run --env CODECLIMATE_CODE="$PWD" --volume "$PWD":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate:0.69.0 analyze -f json > codeclimate.json || true
artifacts:
paths: [codeclimate.json]
```
[cli]: https://github.com/codeclimate/codeclimate
[dind]: ../docker/using_docker_build.md#use-docker-in-docker-executor
[ee]: https://about.gitlab.com/pricing/
doc/ci/examples/container_scanning.md
View file @ 858d093b
# Container Scanning with GitLab CI/CD
CAUTION: **Caution:**
The job definition shown below is supported on GitLab 11.5 and later versions.
It also requires the GitLab Runner 11.5 or later.
For earlier versions, use the [previous job definitions](#previous-job-definitions).
You can check your Docker images (or more precisely the containers) for known
vulnerabilities by using [Clair](https://github.com/coreos/clair) and
[clair-scanner](https://github.com/arminc/clair-scanner), two open source tools
for Vulnerability Static Analysis for containers.
All you need is a GitLab Runner with the Docker executor (the shared Runners on
GitLab.com will work fine). You can then add a new job to `.gitlab-ci.yml`,
called `container_scanning`:
First, you need GitLab Runner with
[docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-executor).
Once you set up the Runner, add a new job to `.gitlab-ci.yml` that
generates the expected report:
```yaml
container_scanning:
......@@ -36,32 +43,26 @@ container_scanning:
- while( ! wget -T 10 -q -O /dev/null http://docker:6060/v1/namespaces ) ; do sleep 1 ; echo -n "." ; if [ $retries -eq 10 ] ; then echo " Timeout, aborting." ; exit 1 ; fi ; retries=$(($retries+1)) ; done
- ./clair-scanner -c http://docker:6060 --ip $(hostname -i) -r gl-container-scanning-report.json -l clair.log -w clair-whitelist.yml ${CI_APPLICATION_REPOSITORY}:${CI_APPLICATION_TAG} || true
artifacts:
paths: [gl-container-scanning-report.json]
reports:
container_scanning: gl-container-scanning-report.json
```
The above example will create a `container_scanning` job in your CI/CD pipeline, pull
the image from the [Container Registry](../../user/project/container_registry.md)
(whose name is defined from the two `CI_APPLICATION_` variables) and scan it
for possible vulnerabilities. The report will be saved as an artifact that you
can later download and analyze.
for possible vulnerabilities. The report will be saved as a
[Container Scanning report artifact](https://docs.gitlab.com/ee//ci/yaml/README.html#artifactsreportscontainer_scanning)
that you can later download and analyze.
Due to implementation limitations we always take the latest Container Scanning artifact available.
If you want to whitelist some specific vulnerabilities, you can do so by defining
them in a [YAML file](https://github.com/arminc/clair-scanner/blob/master/README.md#example-whitelist-yaml-file),
in our case its named `clair-whitelist.yml`.
TIP: **Tip:**
Starting with [GitLab Ultimate][ee] 10.4, this information will
be automatically extracted and shown right in the merge request widget. To do
so, the CI/CD job must be named `container_scanning` and the artifact path must be
`gl-container-scanning-report.json`.
[Learn more on container scanning results shown in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/container_scanning.html).
CAUTION: **Caution:**
Before GitLab 11.0, Container Scanning was previously using `sast:container` for job name and
`gl-sast-container-report.json` for the artifact name. While these old names
are still maintained, they have been deprecated with GitLab 11.0 and may be removed
in next major release, GitLab 12.0. You are advised to update your current `.gitlab-ci.yml`
configuration to reflect that change.
For [GitLab Ultimate][ee] users, this information will
be automatically extracted and shown right in the merge request widget.
[Learn more on Container Scanning in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/container_scanning.html).
CAUTION: **Caution:**
Starting with GitLab 11.5, Container Scanning feature is licensed under the name `container_scanning`.
......@@ -69,4 +70,50 @@ While the old name `sast_container` is still maintained, it has been deprecated
may be removed in next major release, GitLab 12.0. You are advised to update your current `.gitlab-ci.yml`
configuration to reflect that change if you are using the `$GITLAB_FEATURES` environment variable.
## Previous job definitions
CAUTION: **Caution:**
Before GitLab 11.5, Container Scanning job and artifact had to be named specifically
to automatically extract report data and show it in the merge request widget.
While these old job definitions are still maintained they have been deprecated
and may be removed in next major release, GitLab 12.0.
You are advised to update your current `.gitlab-ci.yml` configuration to reflect that change.
For GitLab 11.4 and earlier, the job should look like:
```yaml
container_scanning:
image: docker:stable
variables:
DOCKER_DRIVER: overlay2
## Define two new variables based on GitLab's CI/CD predefined variables
## https://docs.gitlab.com/ee/ci/variables/#predefined-variables-environment-variables
CI_APPLICATION_REPOSITORY: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG
CI_APPLICATION_TAG: $CI_COMMIT_SHA
allow_failure: true
services:
- docker:stable-dind
script:
- docker run -d --name db arminc/clair-db:latest
- docker run -p 6060:6060 --link db:postgres -d --name clair --restart on-failure arminc/clair-local-scan:v2.0.1
- apk add -U wget ca-certificates
- docker pull ${CI_APPLICATION_REPOSITORY}:${CI_APPLICATION_TAG}
- wget https://github.com/arminc/clair-scanner/releases/download/v8/clair-scanner_linux_amd64
- mv clair-scanner_linux_amd64 clair-scanner
- chmod +x clair-scanner
- touch clair-whitelist.yml
- while( ! wget -q -O /dev/null http://docker:6060/v1/namespaces ) ; do sleep 1 ; done
- retries=0
- echo "Waiting for clair daemon to start"
- while( ! wget -T 10 -q -O /dev/null http://docker:6060/v1/namespaces ) ; do sleep 1 ; echo -n "." ; if [ $retries -eq 10 ] ; then echo " Timeout, aborting." ; exit 1 ; fi ; retries=$(($retries+1)) ; done
- ./clair-scanner -c http://docker:6060 --ip $(hostname -i) -r gl-container-scanning-report.json -l clair.log -w clair-whitelist.yml ${CI_APPLICATION_REPOSITORY}:${CI_APPLICATION_TAG} || true
artifacts:
paths: [gl-container-scanning-report.json]
```
Alternatively the job name could be `sast:container`
and the artifact name could be `gl-sast-container-report.json`.
These names have been deprecated with GitLab 11.0
and may be removed in next major release, GitLab 12.0.
[ee]: https://about.gitlab.com/pricing/
doc/ci/examples/dast.md
View file @ 858d093b
# Dynamic Application Security Testing with GitLab CI/CD
CAUTION: **Caution:**
The job definition shown below is supported on GitLab 11.5 and later versions.
It also requires the GitLab Runner 11.5 or later.
For earlier versions, use the [previous job definitions](#previous-job-definitions).
[Dynamic Application Security Testing (DAST)](https://en.wikipedia.org/wiki/Dynamic_program_analysis)
is using the popular open source tool [OWASP ZAProxy](https://github.com/zaproxy/zaproxy)
to perform an analysis on your running web application.
Since it is based on [ZAP Baseline](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan)
DAST will perform passive scanning only;
it will not actively attack your application.
It can be very useful combined with [Review Apps](../review_apps/index.md).
## Example
All you need is a GitLab Runner with the Docker executor (the shared Runners on
GitLab.com will work fine). You can then add a new job to `.gitlab-ci.yml`,
called `dast`:
First, you need GitLab Runner with
[docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-executor).
Once you set up the Runner, add a new job to `.gitlab-ci.yml` that
generates the expected report:
```yaml
dast:
......@@ -23,13 +33,16 @@ dast:
- /zap/zap-baseline.py -J gl-dast-report.json -t $website || true
- cp /zap/wrk/gl-dast-report.json .
artifacts:
paths: [gl-dast-report.json]
reports:
dast: gl-dast-report.json
```
The above example will create a `dast` job in your CI/CD pipeline which will run
the tests on the URL defined in the `website` variable (change it to use your
own) and finally write the results in the `gl-dast-report.json` file. You can
then download and analyze the report artifact in JSON format.
own) and scan it for possible vulnerabilities. The report will be saved as a
[DAST report artifact](https://docs.gitlab.com/ee//ci/yaml/README.html#artifactsreportsdast)
that you can later download and analyze.
Due to implementation limitations we always take the latest DAST artifact available.
It's also possible to authenticate the user before performing DAST checks:
......@@ -39,25 +52,51 @@ dast:
variables:
website: "https://example.com"
login_url: "https://example.com/sign-in"
username: "john.doe@example.com"
password: "john-doe-password"
allow_failure: true
script:
- mkdir /zap/wrk/
- /zap/zap-baseline.py -J gl-dast-report.json -t $website
--auth-url $login_url
--auth-username "john.doe@example.com"
--auth-password "john-doe-password" || true
--auth-username $username
--auth-password $password || true
- cp /zap/wrk/gl-dast-report.json .
artifacts:
paths: [gl-dast-report.json]
reports:
dast: gl-dast-report.json
```
See [zaproxy documentation](https://gitlab.com/gitlab-org/security-products/zaproxy)
to learn more about authentication settings.
TIP: **Tip:**
Starting with [GitLab Ultimate][ee] 10.4, this information will
be automatically extracted and shown right in the merge request widget. To do
so, the CI job must be named `dast` and the artifact path must be
`gl-dast-report.json`.
[Learn more about DAST results shown in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/dast.html).
For [GitLab Ultimate][ee] users, this information will
be automatically extracted and shown right in the merge request widget.
[Learn more on DAST in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/dast.html).
## Previous job definitions
CAUTION: **Caution:**
Before GitLab 11.5, DAST job and artifact had to be named specifically
to automatically extract report data and show it in the merge request widget.
While these old job definitions are still maintained they have been deprecated
and may be removed in next major release, GitLab 12.0.
You are advised to update your current `.gitlab-ci.yml` configuration to reflect that change.
For GitLab 11.4 and earlier, the job should look like:
```yaml
dast:
image: registry.gitlab.com/gitlab-org/security-products/zaproxy
variables:
website: "https://example.com"
allow_failure: true
script:
- mkdir /zap/wrk/
- /zap/zap-baseline.py -J gl-dast-report.json -t $website || true
- cp /zap/wrk/gl-dast-report.json .
artifacts:
paths: [gl-dast-report.json]
```
[ee]: https://about.gitlab.com/pricing/
doc/ci/yaml/README.md
View file @ 858d093b
......@@ -103,7 +103,7 @@ rspec:
- $RSPEC
```
In the example above, the `rspec` job inherits from the `.tests` template job.
In the example above, the `rspec` job inherits from the `.tests` template job.
GitLab will perform a reverse deep merge based on the keys. GitLab will:
- Merge the `rspec` contents into `.tests` recursively.
......@@ -1337,6 +1337,81 @@ concatenated into a single file. Use a filename pattern (`junit: rspec-*.xml`),
an array of filenames (`junit: [rspec-1.xml, rspec-2.xml, rspec-3.xml]`), or a
combination thereof (`junit: [rspec.xml, test-results/TEST-*.xml]`).
#### `artifacts:reports:codequality` **[STARTER]**
> Introduced in GitLab 11.5. Requires GitLab Runner 11.5 and above.
The `codequality` report collects [CodeQuality issues](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality.html)
as artifacts.
The collected Code Quality report will be uploaded to GitLab as an artifact and will
be automatically shown in merge requests.
#### `artifacts:reports:sast` **[ULTIMATE]**
> Introduced in GitLab 11.5. Requires GitLab Runner 11.5 and above.
The `sast` report collects [SAST vulnerabilities](https://docs.gitlab.com/ee/user/project/merge_requests/sast.html)
as artifacts.
The collected SAST report will be uploaded to GitLab as an artifact and will
be automatically shown in merge requests, pipeline view and provide data for security
dashboards.
#### `artifacts:reports:dependency_scanning` **[ULTIMATE]**
> Introduced in GitLab 11.5. Requires GitLab Runner 11.5 and above.
The `dependency_scanning` report collects [Dependency Scanning vulnerabilities](https://docs.gitlab.com/ee/user/project/merge_requests/dependency_scanning.html)
as artifacts.
The collected Dependency Scanning report will be uploaded to GitLab as an artifact and will
be automatically shown in merge requests, pipeline view and provide data for security
dashboards.
#### `artifacts:reports:container_scanning` **[ULTIMATE]**
> Introduced in GitLab 11.5. Requires GitLab Runner 11.5 and above.
The `container_scanning` report collects [Container Scanning vulnerabilities](https://docs.gitlab.com/ee/user/project/merge_requests/container_scanning.html)
as artifacts.
The collected Container Scanning report will be uploaded to GitLab as an artifact and will
be automatically shown in merge requests, pipeline view and provide data for security
dashboards.
#### `artifacts:reports:dast` **[ULTIMATE]**
> Introduced in GitLab 11.5. Requires GitLab Runner 11.5 and above.
The `dast` report collects [DAST vulnerabilities](https://docs.gitlab.com/ee/user/project/merge_requests/dast.html)
as artifacts.
The collected DAST report will be uploaded to GitLab as an artifact and will
be automatically shown in merge requests, pipeline view and provide data for security
dashboards.
#### `artifacts:reports:license_management` **[ULTIMATE]**
> Introduced in GitLab 11.5. Requires GitLab Runner 11.5 and above.
The `license_management` report collects [Licenses](https://docs.gitlab.com/ee/user/project/merge_requests/license_management.html)
as artifacts.
The collected License Management report will be uploaded to GitLab as an artifact and will
be automatically shown in merge requests, pipeline view and provide data for security
dashboards.
#### `artifacts:reports:performance` **[PREMIUM]**
> Introduced in GitLab 11.5. Requires GitLab Runner 11.5 and above.
The `performance` report collects [Performance metrics](https://docs.gitlab.com/ee//user/project/merge_requests/browser_performance_testing.html)
as artifacts.
The collected Performance report will be uploaded to GitLab as an artifact and will
be automatically shown in merge requests.
## `dependencies`
> Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7