Commit 268da973 authored by Sam Kerr's avatar Sam Kerr

Update integration docs

Add a screenshot

Fix image markdown

Fixing bad markdown

More text updates

More text updates

Add more updates to text

Re-add unintentionally deleted file

Update commas (and add extra words to appease danger bot)

Use a SAST scanner for the SAST example

Clarify example for report artifact

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md

Apply suggestion to doc/development/integrations/secure.md
parent 22ef029a
......@@ -2,14 +2,23 @@
Integrating a security scanner into GitLab consists of providing end users
with a [CI job definition](../../ci/yaml/README.md#introduction)
they can add to their CI configuration files, to scan their GitLab projects.
they can add to their CI configuration files to scan their GitLab projects.
This CI job should then output its results in a GitLab-specified format. These results are then
automatically presented in various places in GitLab, such as the Pipeline view, Merge Request
widget, and Security Dashboard.
The scanning job is usually based on a [Docker image](https://docs.docker.com/)
that contains the scanner and all its dependencies in a self-contained environment.
This page documents requirements and guidelines for writing CI jobs implementing a security scanner,
as well as requirements and guidelines for the Docker image itself.
This page documents requirements and guidelines for writing CI jobs that implement a security
scanner, as well as requirements and guidelines for the Docker image.
## Job definition
This section desribes several important fields to add to the security scanner's job
definition file. Full documentation on these and other available fields can be viewed
in the [CI documentation](../../ci/yaml/README.md#image).
### Name
For consistency, scanning jobs should be named after the scanner, in lower case.
......@@ -26,8 +35,8 @@ containing the security scanner.
### Script
The [`script`](../../ci/yaml/README.md#script) keyword
is used to specify the command that the job runs.
Because the `script` cannot be left empty, it must be set to the command that performs the scan.
is used to specify the commands to run the scanner.
Because the `script` entry can't be left empty, it must be set to the command that performs the scan.
It is not possible to rely on the predefined `ENTRYPOINT` and `CMD` of the Docker image
to perform the scan automatically, without passing any command.
......@@ -60,37 +69,34 @@ For example, here is the definition of a SAST job that generates a file named `g
and uploads it as a SAST report:
```yaml
mysec_dependency_scanning:
mysec_sast_scanning:
image: registry.gitlab.com/secure/mysec
artifacts:
reports:
sast: gl-sast-report.json
```
`gl-sast-report.json` is an example file path. See [the Output file section](#output-file) for more details.
It is processed as a SAST report because it is declared as such in the job definition.
Note that `gl-sast-report.json` is an example file path but any other file name can be used. See
[the Output file section](#output-file) for more details. It's processed as a SAST report because
it's declared under the `reports:sast` key in the job definition, not because of the file name.
### Policies
Scanning jobs should be skipped unless the corresponding feature is listed
in the `GITLAB_FEATURES` variable (comma-separated list of values).
So Dependency Scanning, Container Scanning, SAST, and DAST should be skipped
unless `GITLAB_FEATURES` contains `dependency_scanning`, `container_scanning`, `sast`, and `dast`, respectively.
See [GitLab CI/CD predefined variables](../../ci/variables/predefined_variables.md).
Certain GitLab workflows, such as [AutoDevOps](../../topics/autodevops/customize.md#disable-jobs),
define variables to indicate that given scans should be disabled. You can check for this by looking
for variables such as `DEPENDENCY_SCANNING_DISABLED`, `CONTAINER_SCANNING_DISABLED`,
`SAST_DISABLED`, and `DAST_DISABLED`. If appropriate based on the scanner type, you should then
disable running the custom scanner.
Also, scanning jobs should be skipped when the corresponding variable prefixed with `_DISABLED` is present.
See `DEPENDENCY_SCANNING_DISABLED`, `CONTAINER_SCANNING_DISABLED`, `SAST_DISABLED`, and `DAST_DISABLED`
in [Auto DevOps documentation](../../topics/autodevops/customize.md#disable-jobs).
Finally, SAST and Dependency Scanning job definitions should use
`CI_PROJECT_REPOSITORY_LANGUAGES` (comma-separated list of values)
in order to skip the job when the language or technology is not supported.
GitLab also defines a `CI_PROJECT_REPOSITORY_LANGUAGES` variable, which provides the list of
languages in the repo. Depending on this value, your scanner may or may not do something different.
Language detection currently relies on the [`linguist`](https://github.com/github/linguist) Ruby gem.
See [GitLab CI/CD prefined variables](../../ci/variables/predefined_variables.md#variables-reference).
For instance, here is how to skip the Dependency Scanning job `mysec_dependency_scanning`
unless the project repository contains Java source code,
and the `dependency_scanning` feature is enabled:
#### Policy checking example
This example shows how to skip a custom Dependency Scanning job, `mysec_dependency_scanning`, unless
the project repository contains Java source code and the `dependency_scanning` feature is enabled:
```yaml
mysec_dependency_scanning:
......@@ -111,6 +117,8 @@ for a particular branch or when a particular set of files changes.
The Docker image is a self-contained environment that combines
the scanner with all the libraries and tools it depends on.
Packaging your scanner into a Docker image makes its dependencies and configuration always present,
regardless of the individual machine the scanner runs on.
### Image size
......@@ -144,7 +152,7 @@ It also generates text output on the standard output and standard error streams,
All CI variables are passed to the scanner as environment variables.
The scanned project is described by the [predefined CI variables](../../ci/variables/README.md).
#### SAST, Dependency Scanning
#### SAST and Dependency Scanning
SAST and Dependency Scanning scanners must scan the files in the project directory, given by the `CI_PROJECT_DIR` variable.
......@@ -223,11 +231,8 @@ The DAST variant of the report JSON format is not documented at the moment.
### Version
The documentation of
[SAST](../../user/application_security/sast/index.md#reports-json-format),
[Dependency Scanning](../../user/application_security/dependency_scanning/index.md#reports-json-format),
and [Container Scanning](../../user/application_security/container_scanning/index.md#reports-json-format)
describes the Secure report format version.
This field specifies the version of the report schema you are using. Please reference individual scanner
pages for the specific versions to use.
### Vulnerabilities
......@@ -251,12 +256,17 @@ The `id` should not collide with any other scanner another integrator would prov
#### Name, message, and description
The `name` and `message` fields contain a short description of the vulnerability,
whereas the `description` field provides more details.
The `name` and `message` fields contain a short description of the vulnerability.
The `description` field provides more details.
The `name` is context-free and contains no information on where the vulnerability has been found,
The `name` field is context-free and contains no information on where the vulnerability has been found,
whereas the `message` may repeat the location.
As a visual example, this screenshot highlights where these fields are used when viewing a
vulnerability as part of a pipeline view.
![Example Vulnerability](example_vuln.png)
For instance, a `message` for a vulnerability
reported by Dependency Scanning gives information on the vulnerable dependency,
which is redundant with the `location` field of the vulnerability.
......@@ -288,21 +298,17 @@ It should not repeat the other fields of the vulnerability object.
In particular, the `description` should not repeat the `location` (what is affected)
or the `solution` (how to mitigate the risk).
There is a proposal to remove either the `name` or the `message`, to remove ambiguities.
See [issue #36779](https://gitlab.com/gitlab-org/gitlab/issues/36779).
#### Solution
The `solution` field may contain instructions users should follow to fix the vulnerability or to mitigate the risk.
It is intended for users whereas the `remediations` objects are processed automatically by GitLab.
You can use the `solution` field to instruct users how to fix the identified vulnerability or to mitigate
the risk. End-users interact with this field, whereas GitLab automatically processes the
`remediations` objects.
#### Identifiers
The `identifiers` array describes the vulnerability flaw that has been detected.
An identifier object has a `type` and a `value`;
these technical fields are used to tell if two identifiers are the same.
It also has a `name` and a `url`;
these fields are used to display the identifier in the user interface.
The `identifiers` array describes the detected vulnerability. An identifier object's `type` and
`value` fields are used to tell if two identifiers are the same. The user interface uses the
object's `name` and `url` fields to display the identifier.
It is recommended to reuse the identifiers the GitLab scanners already define:
......@@ -316,18 +322,15 @@ It is recommended to reuse the identifiers the GitLab scanners already define:
| [RHSA](https://access.redhat.com/errata/#/) | `rhsa` | RHSA-2020:0111 |
| [ELSA](https://linux.oracle.com/security/) | `elsa` | ELSA-2020-0085 |
The generic identifiers listed above are defined in the [common library](https://gitlab.com/gitlab-org/security-products/analyzers/common);
this library is shared by the analyzers maintained by GitLab,
and this is where you can [contribute](https://gitlab.com/gitlab-org/security-products/analyzers/common/blob/master/issue/identifier.go) new generic identifiers.
Analyzers may also produce vendor-specific or product-specific identifiers;
these do not belong to the [common library](https://gitlab.com/gitlab-org/security-products/analyzers/common).
The generic identifiers listed above are defined in the [common library](https://gitlab.com/gitlab-org/security-products/analyzers/common),
which is shared by the analyzers that GitLab maintains. You can [contribute](https://gitlab.com/gitlab-org/security-products/analyzers/common/blob/master/issue/identifier.go)
new generic identifiers to if needed. Analyzers may also produce vendor-specific or product-specific
identifiers, which don't belong in the [common library](https://gitlab.com/gitlab-org/security-products/analyzers/common).
The first item of the `identifiers` array is called the primary identifier.
The primary identifier is particularly important, because it is used to
[track vulnerabilities](#tracking-merging-vulnerabilities)
as new commits are pushed to the repository.
Identifiers are used to [merge duplicate vulnerabilities](#tracking-merging-vulnerabilities)
[track vulnerabilities](#tracking-and-merging-vulnerabilities) as new commits are pushed to the repository.
Identifiers are also used to [merge duplicate vulnerabilities](#tracking-and-merging-vulnerabilities)
reported for the same commit, except for `CWE` and `WASC`.
### Location
......@@ -336,7 +339,7 @@ The `location` indicates where the vulnerability has been detected.
The format of the location depends on the type of scanning.
Internally GitLab extracts some attributes of the `location` to generate the **location fingerprint**,
which is used to [track vulnerabilities](#tracking-merging-vulnerabilities)
which is used to track vulnerabilities
as new commits are pushed to the repository.
The attributes used to generate the location fingerprint also depend on the type of scanning.
......@@ -426,12 +429,12 @@ combines `file`, `start_line`, and `end_line`,
so these attributes are mandatory.
All other attributes are optional.
### Tracking, merging vulnerabilities
### Tracking and merging vulnerabilities
Users may give feedback on a vulnerability:
- they may dismiss a vulnerability if it does not apply to their projects
- or they may create an issue for a vulnerability, if there is a possible threat
- They may dismiss a vulnerability if it doesn't apply to their projects
- They may create an issue for a vulnerability if there's a possible threat
GitLab tracks vulnerabilities so that user feedback is not lost
when new Git commits are pushed to the repository.
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment