Merge branch '5483_add_license_management' into 'master'

Expose license management reports to the MR widget.

See merge request gitlab-org/gitlab-ee!5562

doc/ci/examples/README.md

@@ -45,6 +45,10 @@ There's also a collection of repositories with [example projects](https://gitlab
 
 [Analyze code quality with the Code Climate CLI](code_climate.md).
 
+## Dependencies license management
+
+[Find and manage the licenses of your dependencies](license_management.md).
+
 ## Static Application Security Testing (SAST) **[ULTIMATE]**
 
 [Scan your code for vulnerabilities](sast.md)

doc/ci/examples/license_management.md

+# Dependencies license management with GitLab CI/CD
+
+NOTE: **Note:**
+In order to use this tool, a [GitLab Ultimate][ee] license
+is needed.
+
+This example shows how to run the License Management tool on your
+project's dependencies by using GitLab CI/CD.
+
+First, you need GitLab Runner with [docker-in-docker executor](https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-in-docker-executor).
+You can then add a new job to `.gitlab-ci.yml`, called `license_management`:
+
+```yaml
+license_management:
+  image: registry.gitlab.com/gitlab-org/security-products/license-management:latest
+  allow_failure: true
+  script:
+    - /run.sh .
+  artifacts:
+    paths: [gl-license-report.json]
+```
+
+The above example will create a `license_management` job in the `test` stage and will create the required report artifact. Check the
+[Auto-DevOps template](https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Auto-DevOps.gitlab-ci.yml)
+for a full reference.
+
+
+TIP: **Tip:**
+Starting with [GitLab Ultimate][ee] 10.8, this information will
+be automatically extracted and shown right in the merge request widget. To do
+so, the CI job must be named `license_management` and the artifact path must be
+`gl-license-report.json`. Make sure your pipeline has a stage nammed `test`,
+or specify another existing stage inside the `license_management` job.
+[Learn more on license management results shown in merge requests](../../user/project/merge_requests/license_management.md).
+
+
+[ee]: https://about.gitlab.com/products/

doc/topics/autodevops/index.md

@@ -43,6 +43,7 @@ project in an easy and automatic way:
 1. [Auto Code Quality](#auto-code-quality)
 1. [Auto SAST (Static Application Security Testing)](#auto-sast)
 1. [Auto Dependency Scanning](#auto-dependency-scanning)
+1. [Auto License Management](#auto-license-management)
 1. [Auto Container Scanning](#auto-container-scanning)
 1. [Auto Review Apps](#auto-review-apps)
 1. [Auto DAST (Dynamic Application Security Testing)](#auto-dast)
@@ -258,6 +259,19 @@ check out.
 In GitLab Ultimate, any security warnings are also
 [shown in the merge request widget](../../user/project/merge_requests/dependency_scanning.md).
 
+### Auto License Management **[ULTIMATE]**
+
+> Introduced in [GitLab Ultimate][ee] 10.8.
+
+License Management uses the
+[License Management Docker image](https://gitlab.com/gitlab-org/security-products/license_management)
+to search the project dependencies for their license. Once the
+report is created, it's uploaded as an artifact which you can later download and
+check out.
+
+In GitLab Ultimate, any license are also
+[shown in the merge request widget](../../user/project/merge_requests/license_management.md).
+
 ### Auto Container Scanning
 
 > Introduced in GitLab 10.4.

doc/user/project/merge_requests/index.md

@@ -38,6 +38,7 @@ With **[GitLab Enterprise Edition][ee]**, you can also:
 - Analyze the impact of your changes with [Code Quality reports](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html) **[STARTER]**
 - Analyze your source code for vulnerabilities with [Static Application Security Testing](#static-application-security-testing) **[ULTIMATE]**
 - Analyze your dependencies for vulnerabilities with [Dependency Scanning](#dependency-scanning) **[ULTIMATE]**
+- Manage the licenses of your dependencies with [License Management](#license-management) **[ULTIMATE]**
 - Analyze your Docker images for vulnerabilities with [Container Scanning](#container-scanning) **[ULTIMATE]**
 - Analyze your running web applications for vulnerabilities with [Dynamic Application Security Testing](#dynamic-application-security-testing) **[ULTIMATE]**
 - Determine the performance impact of changes with [Browser Performance Testing](#browser-performance-testing) **[PREMIUM]**
@@ -49,6 +50,7 @@ A. Consider you are a software developer working in a team:
 1. You checkout a new branch, and submit your changes through a merge request
 1. You gather feedback from your team
 1. You work on the implementation optimizing code with [Code Quality reports](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html) **[STARTER]**
+1. You avoid using dependencies whose license is not compatible with your project with [License Management reports](https://docs.gitlab.com/ee/user/project/merge_requests/license-management.html) **[ULTIMATE]**
 1. You build and test your changes with GitLab CI/CD
 1. You request the [approval](#merge-request-approvals) from your manager
 1. Your manager pushes a commit with his final review, [approves the merge request](#merge-request-approvals), and set it to [merge when pipeline succeeds](#merge-when-pipeline-succeeds)
@@ -259,6 +261,17 @@ merge request widget area.
 
 [Read more about Dependency Scanning reports.](dependency_scanning.md)
 
+## License Management **[ULTIMATE]**
+
+> Introduced in [GitLab Ultimate][products] 10.8.
+
+If you are using [GitLab CI/CD][ci], you can search your dependencies for their 
+licenses using License Management.
+Going a step further, GitLab can show the licenses report right in the
+merge request widget area.
+
+[Read more about License Management reports.](license_management.md)
+
 ## Container Scanning **[ULTIMATE]**
 
 > Introduced in [GitLab Ultimate][products] 10.4.

doc/user/project/merge_requests/license_management.md

+# License Management
+
+> [Introduced][ee-5483] in [GitLab Ultimate][ee] 10.8.
+
+## Overview
+
+If you are using [GitLab CI/CD][ci], you can search your project dependencies for their licenses
+using License Management, either by
+including the CI job in your [existing `.gitlab-ci.yml` file][cc-docs] or
+by implicitly using [Auto License Management](../../../topics/autodevops/index.md#auto-dependency-scanning)
+that is provided by [Auto DevOps](../../../topics/autodevops/index.md).
+
+Going a step further, GitLab can show the licenses list right in the merge
+request widget area.
+
+## Use cases
+
+It helps you find licenses that you don't want to use in your project and see
+which dependencies use them. For example, your application is using an external (open source)
+library whose license is incompatible with yours.
+
+## Supported languages and dependency managers
+
+The following languages and dependency managers are supported.
+
+| Language (package managers)                                                 |
+|-----------------------------------------------------------------------------|
+| Bower ([Bower](https://bower.io/))                                          |
+| Go ([Godep](https://github.com/tools/godep), go get)                        |
+| Java ([Gradle](https://gradle.org/), [Maven](https://maven.apache.org/))    |
+| JavaScript ([npm](https://www.npmjs.com/))                                  |
+| .NET ([Nuget](https://www.nuget.org/))                                      |
+| Python ([pip](https://pip.pypa.io/en/stable/))                              |
+| Ruby ([gem](https://rubygems.org/))                                         |
+
+## How it works
+
+First of all, you need to define a job named `license_management` in your
+`.gitlab-ci.yml` file. [Check how the `license_management` job should look like][cc-docs].
+
+In order for the report to show in the merge request, there are two
+prerequisites:
+
+- the specified job **must** be named `license_management`
+- the resulting report **must** be named `gl-license-report.json`
+  and uploaded as an artifact
+
+The `license_management` job will search the application dependencies for licenses,
+the resulting JSON file will be uploaded as an artifact, and
+GitLab will then check this file and show the information inside the merge
+request.
+
+![License Management Widget](img/license_management.jpg)
+
+[ee-5483]: https://gitlab.com/gitlab-org/gitlab-ee/issues/5483
+[ee]: https://about.gitlab.com/products/
+[ci]: ../../../ci/README.md
+[cc-docs]: ../../../ci/examples/license_management.md

ee/app/helpers/ee/gitlab_routing_helper.rb

@@ -39,5 +39,11 @@ module EE
                                       pipeline.dependency_scanning_artifact,
                                       path: Ci::Build::DEPENDENCY_SCANNING_FILE)
     end
+
+    def license_management_artifact_url(pipeline)
+      raw_project_build_artifacts_url(pipeline.project,
+                                      pipeline.license_management_artifact,
+                                      path: Ci::Build::LICENSE_MANAGEMENT_FILE)
+    end
   end
 end

ee/app/models/ee/ci/build.rb

@@ -9,6 +9,7 @@ module EE
 
       CODEQUALITY_FILE = 'codeclimate.json'.freeze
       DEPENDENCY_SCANNING_FILE = 'gl-dependency-scanning-report.json'.freeze
+      LICENSE_MANAGEMENT_FILE = 'gl-license-report.json'.freeze
       SAST_FILE = 'gl-sast-report.json'.freeze
       PERFORMANCE_FILE = 'performance.json'.freeze
       SAST_CONTAINER_FILE = 'gl-sast-container-report.json'.freeze
@@ -19,6 +20,7 @@ module EE
         scope :performance, -> { where(name: %w[performance deploy]) }
         scope :sast, -> { where(name: 'sast') }
         scope :dependency_scanning, -> { where(name: 'dependency_scanning') }
+        scope :license_management, -> { where(name: 'license_management') }
         scope :sast_container, -> { where(name: %w[container_scanning sast:container]) }
         scope :dast, -> { where(name: 'dast') }
 
@@ -58,6 +60,10 @@ module EE
         has_artifact?(DEPENDENCY_SCANNING_FILE)
       end
 
+      def has_license_management_json?
+        has_artifact?(LICENSE_MANAGEMENT_FILE)
+      end
+
       def has_sast_container_json?
         has_artifact?(SAST_CONTAINER_FILE)
       end

ee/app/models/ee/ci/pipeline.rb

@@ -28,6 +28,10 @@ module EE
         @dependency_scanning_artifact ||= artifacts.dependency_scanning.find(&:has_dependency_scanning_json?)
       end
 
+      def license_management_artifact
+        @license_management_artifact ||= artifacts.license_management.find(&:has_license_management_json?)
+      end
+
       def sast_container_artifact
         @sast_container_artifact ||= artifacts.sast_container.find(&:has_sast_container_json?)
       end
@@ -48,6 +52,10 @@ module EE
         dependency_scanning_artifact&.success?
       end
 
+      def has_license_management_data?
+        license_management_artifact&.success?
+      end
+
       def has_sast_container_data?
         sast_container_artifact&.success?
       end
@@ -74,6 +82,11 @@ module EE
           has_dependency_scanning_data?
       end
 
+      def expose_license_management_data?
+        project.feature_available?(:license_management) &&
+          has_license_management_data?
+      end
+
       def expose_sast_container_data?
         project.feature_available?(:sast_container) &&
           has_sast_container_data?

ee/app/models/ee/merge_request.rb

@@ -18,6 +18,8 @@ module EE
       delegate :sast_artifact, to: :base_pipeline, prefix: :base, allow_nil: true
       delegate :dependency_scanning_artifact, to: :head_pipeline, prefix: :head, allow_nil: true
       delegate :dependency_scanning_artifact, to: :base_pipeline, prefix: :base, allow_nil: true
+      delegate :license_management_artifact, to: :head_pipeline, prefix: :head, allow_nil: true
+      delegate :license_management_artifact, to: :base_pipeline, prefix: :base, allow_nil: true
       delegate :sast_container_artifact, to: :head_pipeline, prefix: :head, allow_nil: true
       delegate :sast_container_artifact, to: :base_pipeline, prefix: :base, allow_nil: true
       delegate :dast_artifact, to: :head_pipeline, prefix: :head, allow_nil: true
@@ -26,10 +28,12 @@ module EE
       delegate :sha, to: :base_pipeline, prefix: :base_pipeline, allow_nil: true
       delegate :has_sast_data?, to: :base_pipeline, prefix: :base, allow_nil: true
       delegate :has_dependency_scanning_data?, to: :base_pipeline, prefix: :base, allow_nil: true
+      delegate :has_license_management_data?, to: :base_pipeline, prefix: :base, allow_nil: true
       delegate :has_sast_container_data?, to: :base_pipeline, prefix: :base, allow_nil: true
       delegate :has_dast_data?, to: :base_pipeline, prefix: :base, allow_nil: true
       delegate :expose_sast_data?, to: :head_pipeline, allow_nil: true
       delegate :expose_dependency_scanning_data?, to: :head_pipeline, allow_nil: true
+      delegate :expose_license_management_data?, to: :head_pipeline, allow_nil: true
       delegate :expose_sast_container_data?, to: :head_pipeline, allow_nil: true
       delegate :expose_dast_data?, to: :head_pipeline, allow_nil: true
     end

ee/app/models/license.rb

@@ -64,6 +64,7 @@ class License < ActiveRecord::Base
 
   EEU_FEATURES = EEP_FEATURES + %i[
     dependency_scanning
+    license_management
     sast
     sast_container
     cluster_health

ee/app/serializers/ee/merge_request_widget_entity.rb

@@ -69,6 +69,20 @@ module EE
         end
       end
 
+      expose :license_management, if: -> (mr, _) { mr.expose_license_management_data? } do
+        expose :head_path, if: -> (mr, _) { can?(current_user, :read_build, mr.head_license_management_artifact) } do |merge_request|
+          raw_project_build_artifacts_url(merge_request.source_project,
+                                          merge_request.head_license_management_artifact,
+                                          path: Ci::Build::LICENSE_MANAGEMENT_FILE)
+        end
+
+        expose :base_path, if: -> (mr, _) { mr.base_has_license_management_data? && can?(current_user, :read_build, mr.base_license_management_artifact) } do |merge_request|
+          raw_project_build_artifacts_url(merge_request.target_project,
+                                          merge_request.base_license_management_artifact,
+                                          path: Ci::Build::LICENSE_MANAGEMENT_FILE)
+        end
+      end
+
       expose :sast_container, if: -> (mr, _) { mr.expose_sast_container_data? } do
         expose :head_path, if: -> (mr, _) { can?(current_user, :read_build, mr.head_sast_container_artifact) } do |merge_request|
           raw_project_build_artifacts_url(merge_request.source_project,

ee/spec/models/ci/build_spec.rb

@@ -142,6 +142,7 @@ describe Ci::Build do
     has_performance_json?: Ci::Build::PERFORMANCE_FILE,
     has_sast_json?: Ci::Build::SAST_FILE,
     has_dependency_scanning_json?: Ci::Build::DEPENDENCY_SCANNING_FILE,
+    has_license_management_json?: Ci::Build::LICENSE_MANAGEMENT_FILE,
     has_sast_container_json?: Ci::Build::SAST_CONTAINER_FILE,
     has_dast_json?: Ci::Build::DAST_FILE
   }.freeze

ee/spec/models/ci/pipeline_spec.rb

@@ -22,6 +22,7 @@ describe Ci::Pipeline do
     performance_artifact: [Ci::Build::PERFORMANCE_FILE, 'performance'],
     sast_artifact: [Ci::Build::SAST_FILE, 'sast'],
     dependency_scanning_artifact: [Ci::Build::DEPENDENCY_SCANNING_FILE, 'dependency_scanning'],
+    license_management_artifact: [Ci::Build::LICENSE_MANAGEMENT_FILE, 'license_management'],
     sast_container_artifact: [Ci::Build::SAST_CONTAINER_FILE, 'container_scanning'],
     dast_artifact: [Ci::Build::DAST_FILE, 'dast']
   }.freeze

ee/spec/serializers/merge_request_widget_entity_spec.rb

@@ -60,6 +60,19 @@ describe MergeRequestWidgetEntity do
     expect(subject.as_json[:dependency_scanning]).to include(:base_path)
   end
 
+  it 'has license_management data' do
+    build = create(:ci_build, name: 'license_management', pipeline: pipeline)
+
+    allow(merge_request).to receive(:expose_license_management_data?).and_return(true)
+    allow(merge_request).to receive(:base_has_license_management_data?).and_return(true)
+    allow(merge_request).to receive(:base_license_management_artifact).and_return(build)
+    allow(merge_request).to receive(:head_license_management_artifact).and_return(build)
+
+    expect(subject.as_json).to include(:license_management)
+    expect(subject.as_json[:license_management]).to include(:head_path)
+    expect(subject.as_json[:license_management]).to include(:base_path)
+  end
+
   it 'has sast_container data' do
     build = create(:ci_build, name: 'sast:image', pipeline: pipeline)