Commit 268da973 authored by Sam Kerr's avatar Sam Kerr

Update integration docs

Add a screenshot

Fix image markdown

Fixing bad markdown

More text updates

More text updates

Add more updates to text

Re-add unintentionally deleted file

Update commas (and add extra words to appease danger bot)

Use a SAST scanner for the SAST example

Clarify example for report artifact

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md
parent 22ef029a
...@@ -2,14 +2,23 @@ ...@@ -2,14 +2,23 @@
Integrating a security scanner into GitLab consists of providing end users Integrating a security scanner into GitLab consists of providing end users
with a [CI job definition](../../ci/yaml/README.md#introduction) with a [CI job definition](../../ci/yaml/README.md#introduction)
they can add to their CI configuration files, to scan their GitLab projects. they can add to their CI configuration files to scan their GitLab projects.
This CI job should then output its results in a GitLab-specified format. These results are then
automatically presented in various places in GitLab, such as the Pipeline view, Merge Request
widget, and Security Dashboard.
The scanning job is usually based on a [Docker image](https://docs.docker.com/) The scanning job is usually based on a [Docker image](https://docs.docker.com/)
that contains the scanner and all its dependencies in a self-contained environment. that contains the scanner and all its dependencies in a self-contained environment.
This page documents requirements and guidelines for writing CI jobs implementing a security scanner,
as well as requirements and guidelines for the Docker image itself. This page documents requirements and guidelines for writing CI jobs that implement a security
scanner, as well as requirements and guidelines for the Docker image.
## Job definition ## Job definition
This section desribes several important fields to add to the security scanner's job
definition file. Full documentation on these and other available fields can be viewed
in the [CI documentation](../../ci/yaml/README.md#image).
### Name ### Name
For consistency, scanning jobs should be named after the scanner, in lower case. For consistency, scanning jobs should be named after the scanner, in lower case.
...@@ -26,8 +35,8 @@ containing the security scanner. ...@@ -26,8 +35,8 @@ containing the security scanner.
### Script ### Script
The [`script`](../../ci/yaml/README.md#script) keyword The [`script`](../../ci/yaml/README.md#script) keyword
is used to specify the command that the job runs. is used to specify the commands to run the scanner.
Because the `script` cannot be left empty, it must be set to the command that performs the scan. Because the `script` entry can't be left empty, it must be set to the command that performs the scan.
It is not possible to rely on the predefined `ENTRYPOINT` and `CMD` of the Docker image It is not possible to rely on the predefined `ENTRYPOINT` and `CMD` of the Docker image
to perform the scan automatically, without passing any command. to perform the scan automatically, without passing any command.
...@@ -60,37 +69,34 @@ For example, here is the definition of a SAST job that generates a file named `g ...@@ -60,37 +69,34 @@ For example, here is the definition of a SAST job that generates a file named `g
and uploads it as a SAST report: and uploads it as a SAST report:
```yaml ```yaml
mysec_dependency_scanning: mysec_sast_scanning:
image: registry.gitlab.com/secure/mysec image: registry.gitlab.com/secure/mysec
artifacts: artifacts:
reports: reports:
sast: gl-sast-report.json sast: gl-sast-report.json
``` ```
`gl-sast-report.json` is an example file path. See [the Output file section](#output-file) for more details. Note that `gl-sast-report.json` is an example file path but any other file name can be used. See
It is processed as a SAST report because it is declared as such in the job definition. [the Output file section](#output-file) for more details. It's processed as a SAST report because
it's declared under the `reports:sast` key in the job definition, not because of the file name.
### Policies ### Policies
Scanning jobs should be skipped unless the corresponding feature is listed Certain GitLab workflows, such as [AutoDevOps](../../topics/autodevops/customize.md#disable-jobs),
in the `GITLAB_FEATURES` variable (comma-separated list of values). define variables to indicate that given scans should be disabled. You can check for this by looking
So Dependency Scanning, Container Scanning, SAST, and DAST should be skipped for variables such as `DEPENDENCY_SCANNING_DISABLED`, `CONTAINER_SCANNING_DISABLED`,
unless `GITLAB_FEATURES` contains `dependency_scanning`, `container_scanning`, `sast`, and `dast`, respectively. `SAST_DISABLED`, and `DAST_DISABLED`. If appropriate based on the scanner type, you should then
See [GitLab CI/CD predefined variables](../../ci/variables/predefined_variables.md). disable running the custom scanner.
Also, scanning jobs should be skipped when the corresponding variable prefixed with `_DISABLED` is present. GitLab also defines a `CI_PROJECT_REPOSITORY_LANGUAGES` variable, which provides the list of
See `DEPENDENCY_SCANNING_DISABLED`, `CONTAINER_SCANNING_DISABLED`, `SAST_DISABLED`, and `DAST_DISABLED` languages in the repo. Depending on this value, your scanner may or may not do something different.
in [Auto DevOps documentation](../../topics/autodevops/customize.md#disable-jobs).
Finally, SAST and Dependency Scanning job definitions should use
`CI_PROJECT_REPOSITORY_LANGUAGES` (comma-separated list of values)
in order to skip the job when the language or technology is not supported.
Language detection currently relies on the [`linguist`](https://github.com/github/linguist) Ruby gem. Language detection currently relies on the [`linguist`](https://github.com/github/linguist) Ruby gem.
See [GitLab CI/CD prefined variables](../../ci/variables/predefined_variables.md#variables-reference). See [GitLab CI/CD prefined variables](../../ci/variables/predefined_variables.md#variables-reference).
For instance, here is how to skip the Dependency Scanning job `mysec_dependency_scanning` #### Policy checking example
unless the project repository contains Java source code,
and the `dependency_scanning` feature is enabled: This example shows how to skip a custom Dependency Scanning job, `mysec_dependency_scanning`, unless
the project repository contains Java source code and the `dependency_scanning` feature is enabled:
```yaml ```yaml
mysec_dependency_scanning: mysec_dependency_scanning:
...@@ -111,6 +117,8 @@ for a particular branch or when a particular set of files changes. ...@@ -111,6 +117,8 @@ for a particular branch or when a particular set of files changes.
The Docker image is a self-contained environment that combines The Docker image is a self-contained environment that combines
the scanner with all the libraries and tools it depends on. the scanner with all the libraries and tools it depends on.
Packaging your scanner into a Docker image makes its dependencies and configuration always present,
regardless of the individual machine the scanner runs on.
### Image size ### Image size
...@@ -144,7 +152,7 @@ It also generates text output on the standard output and standard error streams, ...@@ -144,7 +152,7 @@ It also generates text output on the standard output and standard error streams,
All CI variables are passed to the scanner as environment variables. All CI variables are passed to the scanner as environment variables.
The scanned project is described by the [predefined CI variables](../../ci/variables/README.md). The scanned project is described by the [predefined CI variables](../../ci/variables/README.md).
#### SAST, Dependency Scanning #### SAST and Dependency Scanning
SAST and Dependency Scanning scanners must scan the files in the project directory, given by the `CI_PROJECT_DIR` variable. SAST and Dependency Scanning scanners must scan the files in the project directory, given by the `CI_PROJECT_DIR` variable.
...@@ -223,11 +231,8 @@ The DAST variant of the report JSON format is not documented at the moment. ...@@ -223,11 +231,8 @@ The DAST variant of the report JSON format is not documented at the moment.
### Version ### Version
The documentation of This field specifies the version of the report schema you are using. Please reference individual scanner
[SAST](../../user/application_security/sast/index.md#reports-json-format), pages for the specific versions to use.
[Dependency Scanning](../../user/application_security/dependency_scanning/index.md#reports-json-format),
and [Container Scanning](../../user/application_security/container_scanning/index.md#reports-json-format)
describes the Secure report format version.
### Vulnerabilities ### Vulnerabilities
...@@ -251,12 +256,17 @@ The `id` should not collide with any other scanner another integrator would prov ...@@ -251,12 +256,17 @@ The `id` should not collide with any other scanner another integrator would prov
#### Name, message, and description #### Name, message, and description
The `name` and `message` fields contain a short description of the vulnerability, The `name` and `message` fields contain a short description of the vulnerability.
whereas the `description` field provides more details. The `description` field provides more details.
The `name` is context-free and contains no information on where the vulnerability has been found, The `name` field is context-free and contains no information on where the vulnerability has been found,
whereas the `message` may repeat the location. whereas the `message` may repeat the location.
As a visual example, this screenshot highlights where these fields are used when viewing a
vulnerability as part of a pipeline view.
![Example Vulnerability](example_vuln.png)
For instance, a `message` for a vulnerability For instance, a `message` for a vulnerability
reported by Dependency Scanning gives information on the vulnerable dependency, reported by Dependency Scanning gives information on the vulnerable dependency,
which is redundant with the `location` field of the vulnerability. which is redundant with the `location` field of the vulnerability.
...@@ -288,21 +298,17 @@ It should not repeat the other fields of the vulnerability object. ...@@ -288,21 +298,17 @@ It should not repeat the other fields of the vulnerability object.
In particular, the `description` should not repeat the `location` (what is affected) In particular, the `description` should not repeat the `location` (what is affected)
or the `solution` (how to mitigate the risk). or the `solution` (how to mitigate the risk).
There is a proposal to remove either the `name` or the `message`, to remove ambiguities.
See [issue #36779](https://gitlab.com/gitlab-org/gitlab/issues/36779).
#### Solution #### Solution
The `solution` field may contain instructions users should follow to fix the vulnerability or to mitigate the risk. You can use the `solution` field to instruct users how to fix the identified vulnerability or to mitigate
It is intended for users whereas the `remediations` objects are processed automatically by GitLab. the risk. End-users interact with this field, whereas GitLab automatically processes the
`remediations` objects.
#### Identifiers #### Identifiers
The `identifiers` array describes the vulnerability flaw that has been detected. The `identifiers` array describes the detected vulnerability. An identifier object's `type` and
An identifier object has a `type` and a `value`; `value` fields are used to tell if two identifiers are the same. The user interface uses the
these technical fields are used to tell if two identifiers are the same. object's `name` and `url` fields to display the identifier.
It also has a `name` and a `url`;
these fields are used to display the identifier in the user interface.
It is recommended to reuse the identifiers the GitLab scanners already define: It is recommended to reuse the identifiers the GitLab scanners already define:
...@@ -316,18 +322,15 @@ It is recommended to reuse the identifiers the GitLab scanners already define: ...@@ -316,18 +322,15 @@ It is recommended to reuse the identifiers the GitLab scanners already define:
| [RHSA](https://access.redhat.com/errata/#/) | `rhsa` | RHSA-2020:0111 | | [RHSA](https://access.redhat.com/errata/#/) | `rhsa` | RHSA-2020:0111 |
| [ELSA](https://linux.oracle.com/security/) | `elsa` | ELSA-2020-0085 | | [ELSA](https://linux.oracle.com/security/) | `elsa` | ELSA-2020-0085 |
The generic identifiers listed above are defined in the [common library](https://gitlab.com/gitlab-org/security-products/analyzers/common); The generic identifiers listed above are defined in the [common library](https://gitlab.com/gitlab-org/security-products/analyzers/common),
this library is shared by the analyzers maintained by GitLab, which is shared by the analyzers that GitLab maintains. You can [contribute](https://gitlab.com/gitlab-org/security-products/analyzers/common/blob/master/issue/identifier.go)
and this is where you can [contribute](https://gitlab.com/gitlab-org/security-products/analyzers/common/blob/master/issue/identifier.go) new generic identifiers. new generic identifiers to if needed. Analyzers may also produce vendor-specific or product-specific
Analyzers may also produce vendor-specific or product-specific identifiers; identifiers, which don't belong in the [common library](https://gitlab.com/gitlab-org/security-products/analyzers/common).
these do not belong to the [common library](https://gitlab.com/gitlab-org/security-products/analyzers/common).
The first item of the `identifiers` array is called the primary identifier. The first item of the `identifiers` array is called the primary identifier.
The primary identifier is particularly important, because it is used to The primary identifier is particularly important, because it is used to
[track vulnerabilities](#tracking-merging-vulnerabilities) [track vulnerabilities](#tracking-and-merging-vulnerabilities) as new commits are pushed to the repository.
as new commits are pushed to the repository. Identifiers are also used to [merge duplicate vulnerabilities](#tracking-and-merging-vulnerabilities)
Identifiers are used to [merge duplicate vulnerabilities](#tracking-merging-vulnerabilities)
reported for the same commit, except for `CWE` and `WASC`. reported for the same commit, except for `CWE` and `WASC`.
### Location ### Location
...@@ -336,7 +339,7 @@ The `location` indicates where the vulnerability has been detected. ...@@ -336,7 +339,7 @@ The `location` indicates where the vulnerability has been detected.
The format of the location depends on the type of scanning. The format of the location depends on the type of scanning.
Internally GitLab extracts some attributes of the `location` to generate the **location fingerprint**, Internally GitLab extracts some attributes of the `location` to generate the **location fingerprint**,
which is used to [track vulnerabilities](#tracking-merging-vulnerabilities) which is used to track vulnerabilities
as new commits are pushed to the repository. as new commits are pushed to the repository.
The attributes used to generate the location fingerprint also depend on the type of scanning. The attributes used to generate the location fingerprint also depend on the type of scanning.
...@@ -426,12 +429,12 @@ combines `file`, `start_line`, and `end_line`, ...@@ -426,12 +429,12 @@ combines `file`, `start_line`, and `end_line`,
so these attributes are mandatory. so these attributes are mandatory.
All other attributes are optional. All other attributes are optional.
### Tracking, merging vulnerabilities ### Tracking and merging vulnerabilities
Users may give feedback on a vulnerability: Users may give feedback on a vulnerability:
- they may dismiss a vulnerability if it does not apply to their projects - They may dismiss a vulnerability if it doesn't apply to their projects
- or they may create an issue for a vulnerability, if there is a possible threat - They may create an issue for a vulnerability if there's a possible threat
GitLab tracks vulnerabilities so that user feedback is not lost GitLab tracks vulnerabilities so that user feedback is not lost
when new Git commits are pushed to the repository. when new Git commits are pushed to the repository.
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment