Merge branch 'docs/sast-analyzers' into 'master'

Document all the available options for SAST

Closes gitlab-ee#10120

See merge request gitlab-org/gitlab-ce!30747

doc/user/application_security/sast/analyzers.md

+---
+table_display_block: true
+---
+
+# SAST Analyzers **(ULTIMATE)**
+
+SAST relies on underlying third party tools that are wrapped into what we call
+"Analyzers". An analyzer is a
+[dedicated project](https://gitlab.com/gitlab-org/security-products/analyzers)
+that wraps a particular tool to:
+
+- Expose its detection logic.
+- Handle its execution.
+- Convert its output to the common format.
+
+This is achieved by implementing the [common API](https://gitlab.com/gitlab-org/security-products/analyzers/common).
+
+SAST supports the following official analyzers:
+
+- [Bandit](https://gitlab.com/gitlab-org/security-products/analyzers/bandit)
+- [Brakeman](https://gitlab.com/gitlab-org/security-products/analyzers/brakeman)
+- [ESLint (Javascript)](https://gitlab.com/gitlab-org/security-products/analyzers/eslint)
+- [SpotBugs with the Find Sec Bugs plugin (Ant, Gradle and wrapper, Grails, Maven and wrapper, SBT)](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs)
+- [Flawfinder](https://gitlab.com/gitlab-org/security-products/analyzers/flawfinder)
+- [Gosec](https://gitlab.com/gitlab-org/security-products/analyzers/gosec)
+- [NodeJsScan](https://gitlab.com/gitlab-org/security-products/analyzers/nodejs-scan)
+- [PHP CS security-audit](https://gitlab.com/gitlab-org/security-products/analyzers/phpcs-security-audit)
+- [Secrets (Gitleaks, TruffleHog & Diffence secret detectors)](https://gitlab.com/gitlab-org/security-products/analyzers/secrets)
+- [Security Code Scan (.NET)](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan)
+- [TSLint (Typescript)](https://gitlab.com/gitlab-org/security-products/analyzers/tslint)
+- [Sobelow (Elixir Phoenix)](https://gitlab.com/gitlab-org/security-products/analyzers/sobelow)
+
+The analyzers are published as Docker images that SAST will use to launch
+dedicated containers for each analysis.
+
+SAST is pre-configured with a set of **default images** that are maintained by
+GitLab, but users can also integrate their own **custom images**.
+
+## Official default analyzers
+
+Any custom change to the official analyzers can be achieved by using an
+[environment variable in your `.gitlab-ci.yml`](index.md#customizing-the-sast-settings).
+
+### Using a custom Docker mirror
+
+You can switch to a custom Docker registry that provides the official analyzer
+images under a different prefix. For instance, the following instructs
+SAST to pull `my-docker-registry/gl-images/bandit`
+instead of `registry.gitlab.com/gitlab-org/security-products/analyzers/bandit`.
+In `.gitlab-ci.yml` define:
+
+```yaml
+include:
+  template: SAST.gitlab-ci.yml
+
+variables:
+  SAST_ANALYZER_IMAGE_PREFIX: my-docker-registry/gl-images
+```
+
+This configuration requires that your custom registry provides images for all
+the official analyzers.
+
+### Selecting specific analyzers
+
+You can select the official analyzers you want to run. Here's how to enable
+`bandit` and `flawfinder` while disabling all the other default ones.
+In `.gitlab-ci.yml` define:
+
+```yaml
+include:
+  template: SAST.gitlab-ci.yml
+
+variables:
+  SAST_DEFAULT_ANALYZERS: "bandit,flawfinder"
+```
+
+`bandit` runs first. When merging the reports, SAST will
+remove the duplicates and will keep the `bandit` entries.
+
+### Disabling default analyzers
+
+Setting `SAST_DEFAULT_ANALYZERS` to an empty string will disable all the official
+default analyzers. In `.gitlab-ci.yml` define:
+
+```yaml
+include:
+  template: SAST.gitlab-ci.yml
+
+variables:
+  SAST_DEFAULT_ANALYZERS: ""
+```
+
+That's needed when one totally relies on [custom analyzers](#custom-analyzers).
+
+## Custom Analyzers
+
+You can provide your own analyzers as a comma separated list of Docker images.
+Here's how to add `analyzers/csharp` and `analyzers/perl` to the default images:
+In `.gitlab-ci.yml` define:
+
+```yaml
+include:
+  template: SAST.gitlab-ci.yml
+
+variables:
+  SAST_ANALYZER_IMAGES: "my-docker-registry/analyzers/csharp,amy-docker-registry/analyzers/perl"
+```
+
+The values must be the full path to the container registry images,
+like what you would feed to the `docker pull` command.
+
+NOTE: **Note:**
+This configuration doesn't benefit from the integrated detection step.
+SAST has to fetch and spawn each Docker image to establish whether the
+custom analyzer can scan the source code.
+
+## Analyzers Data
+
+| Property \ Tool                         | Bandit               | Brakeman             | ESLint security      | Find Sec Bugs        | Flawfinder           | Go AST Scanner       | NodeJsScan           | Php CS Security Audit   | Security code Scan (.NET)   | TSLint Security    | Sobelow            |
+| --------------------------------------- | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :---------------------: | :-------------------------: | :-------------:    | :----------------: |
+| Severity                                | ✓   | 𐄂                  | 𐄂                  | ✓   | 𐄂                  | ✓   | 𐄂                  | ✓      | 𐄂                         | ✓ | 𐄂                |
+| Title                                   | ✓   | ✓   | ✓   | ✓   | ✓   | ✓   | ✓   | ✓      | ✓          | ✓ | ✓ |
+| Description                             | 𐄂                  | 𐄂                  | ✓   | ✓   | 𐄂                  | 𐄂                  | ✓   | 𐄂                     | 𐄂                         | ✓ | ✓ |
+| File                                    | ✓   | ✓   | ✓   | ✓   | ✓   | ✓   | ✓   | ✓      | ✓          | ✓ | ✓ |
+| Start line                              | ✓   | ✓   | ✓   | ✓   | ✓   | ✓   | ✓   | ✓      | ✓          | ✓ | ✓ |
+| End line                                | ✓   | 𐄂                  | ✓   | ✓   | 𐄂                  | 𐄂                  | 𐄂                  | 𐄂                     | 𐄂                         | ✓ | 𐄂                |
+| Start column                            | 𐄂                  | 𐄂                  | ✓   | ✓   | ✓   | ✓   | 𐄂                  | ✓      | ✓          | ✓ | 𐄂                |
+| End column                              | 𐄂                  | 𐄂                  | ✓   | ✓   | 𐄂                  | 𐄂                  | 𐄂                  | 𐄂                     | 𐄂                         | ✓ | 𐄂                |
+| External id (e.g. CVE)                  | 𐄂                  | ⚠            | 𐄂                  | ⚠            | ✓   | 𐄂                  | 𐄂                  | 𐄂                     | 𐄂                         | 𐄂                | 𐄂                |
+| URLs                                    | 𐄂                  | ✓   | 𐄂                  | ⚠            | 𐄂                  | 𐄂                  | 𐄂                  | 𐄂                     | 𐄂                         | 𐄂                | 𐄂                |
+| Internal doc/explanation                | ⚠            | ✓   | 𐄂                  | ✓   | 𐄂                  | 𐄂                  | 𐄂                  | 𐄂                     | 𐄂                         | 𐄂                | ✓ |
+| Solution                                | 𐄂                  | 𐄂                  | 𐄂                  | ⚠            | ✓   | 𐄂                  | 𐄂                  | 𐄂                     | 𐄂                         | 𐄂                | 𐄂                |
+| Confidence                              | ✓   | ✓   | 𐄂                  | ✓   | ✓   | ✓   | 𐄂                  | 𐄂                     | 𐄂                         | 𐄂                | ✓ |
+| Affected item (e.g. class or package)   | 𐄂                  | ✓   | 𐄂                  | ✓   | ✓   | 𐄂                  | 𐄂                  | 𐄂                     | 𐄂                         | 𐄂                | 𐄂                |
+| Source code extract                     | ✓   | ✓   | ✓   | 𐄂                  | ✓   | ✓   | 𐄂                  | 𐄂                     | 𐄂                         | 𐄂                | 𐄂                |
+| Internal ID                             | ✓   | ✓   | ✓   | ✓   | ✓   | ✓   | 𐄂                  | ✓      | ✓          | ✓ | ✓ |
+
+- ✓ => we have that data
+- ⚠ => we have that data but it's partially reliable, or we need to extract it from unstructured content
+- 𐄂 => we don't have that data or it would need to develop specific or inefficient/unreliable logic to obtain it.
+
+The values provided by these tools are heterogeneous so they are sometimes
+normalized into common values (e.g., `severity`, `confidence`, etc).

doc/user/application_security/sast/index.md

@@ -135,6 +135,58 @@ sast:
     CI_DEBUG_TRACE: "true"
 ```
 
+### Available variables
+
+SAST can be [configured](#customizing-the-sast-settings) using environment variables.
+
+#### Docker images
+
+The following are Docker image-related variables.
+
+| Environment variable          | Description                                                                    |
+|-------------------------------|--------------------------------------------------------------------------------|
+| `SAST_ANALYZER_IMAGES`        | Comma separated list of custom images. Default images are still enabled. Read more about [customizing analyzers](analyzers.md).       |
+| `SAST_ANALYZER_IMAGE_PREFIX`  | Override the name of the Docker registry providing the default images (proxy). Read more about [customizing analyzers](analyzers.md). |
+| `SAST_ANALYZER_IMAGE_TAG`     | Override the Docker tag of the default images. Read more about [customizing analyzers](analyzers.md).                                 |
+| `SAST_DEFAULT_ANALYZERS`      | Override the names of default images. Read more about [customizing analyzers](analyzers.md).                                          |
+| `SAST_PULL_ANALYZER_IMAGES`   | Pull the images from the Docker registry (set to 0 to disable). Read more about [customizing analyzers](analyzers.md).                 |
+
+### Vulnerability filters
+
+Some analyzers make it possible to filter out vulnerabilities under a given threshold.
+
+| `SAST_BANDIT_EXCLUDED_PATHS`  | -   | comma-separated list of paths to exclude from scan. Uses Python's [`fnmatch` syntax](https://docs.python.org/2/library/fnmatch.html) |
+| `SAST_BRAKEMAN_LEVEL`   |         1 | Ignore Brakeman vulnerabilities under given confidence level. Integer, 1=Low 3=High. |
+| `SAST_FLAWFINDER_LEVEL` |         1 | Ignore Flawfinder vulnerabilities under given risk level. Integer, 0=No risk, 5=High risk. |
+| `SAST_GITLEAKS_ENTROPY_LEVEL` | 8.0 | Minimum entropy for secret detection. Float, 0.0 = low, 8.0 = high. |
+| `SAST_GOSEC_LEVEL`      |         0 | Ignore gosec vulnerabilities under given confidence level. Integer, 0=Undefined, 1=Low, 1=Medium, 3=High. |
+| `SAST_EXCLUDED_PATHS`   |   -        | Exclude vulnerabilities from output based on the paths. This is a comma-separated list of patterns. Patterns can be globs, file or folder paths. Parent directories will also match patterns. |
+
+### Timeouts
+
+The following variables configure timeouts.
+
+| `SAST_DOCKER_CLIENT_NEGOTIATION_TIMEOUT` |      2m | Time limit for Docker client negotiation. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". For example, "300ms", "1.5h" or "2h45m". |
+| `SAST_PULL_ANALYZER_IMAGE_TIMEOUT`       |      5m | Time limit when pulling the image of an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". For example, "300ms", "1.5h" or "2h45m". |
+| `SAST_RUN_ANALYZER_TIMEOUT`              |     20m | Time limit when running an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". For example, "300ms", "1.5h" or "2h45m".|
+
+### Analyzer settings
+
+Some analyzers can be customized with environment variables.
+
+| Environment variable    | Analyzer | Description |
+|-------------------------|----------|----------|
+| `ANT_HOME`              | spotbugs | The `ANT_HOME` environment variable. |
+| `ANT_PATH`              | spotbugs | Path to the `ant` executable. |
+| `GRADLE_PATH`           | spotbugs | Path to the `gradle` executable. |
+| `JAVA_OPTS`             | spotbugs | Additional arguments for the `java` executable. |
+| `JAVA_PATH`             | spotbugs | Path to the `java` executable. |
+| `MAVEN_CLI_OPTS`        | spotbugs | Additional arguments for the `mvn` or `mvnw` executable. |
+| `MAVEN_PATH`            | spotbugs | Path to the `mvn` executable. |
+| `MAVEN_REPO_PATH`       | spotbugs | Path to the Maven local repository (shortcut for the `maven.repo.local` property). |
+| `SBT_PATH`              | spotbugs | Path to the `sbt` executable. |
+| `FAIL_NEVER`            | spotbugs | Set to `1` to ignore compilation failure. |
+
 ## Reports JSON format
 
 CAUTION: **Caution:**