Commit ccc96bb6 authored by Mehmet Emin INAC's avatar Mehmet Emin INAC Committed by Mayra Cabrera

Add `Group Name` & `Project Name` into vulnerability export csv

This will introduce new columns in CSV file for project and instance
level exports called `Group Name` and `Project Name` which might sound
redundant for project exports but simplifys the implementation.
parent 82ee4b7f
......@@ -121,18 +121,18 @@ The response will be `404 Not Found` if the vulnerability export is not finished
Example response:
```csv
Scanner Type,Scanner Name,Status,Vulnerability,Details,Additional Info,Severity,CVE
container_scanning,Clair,confirmed,CVE-2017-16997 in glibc,,CVE-2017-16997 in glibc,critical,CVE-2017-16997
container_scanning,Clair,detected,CVE-2017-18269 in glibc,,CVE-2017-18269 in glibc,critical,CVE-2017-18269
container_scanning,Clair,detected,CVE-2018-1000001 in glibc,,CVE-2018-1000001 in glibc,high,CVE-2018-1000001
container_scanning,Clair,detected,CVE-2016-10228 in glibc,,CVE-2016-10228 in glibc,medium,CVE-2016-10228
container_scanning,Clair,confirmed,CVE-2010-4052 in glibc,,CVE-2010-4052 in glibc,low,CVE-2010-4052
container_scanning,Clair,detected,CVE-2018-18520 in elfutils,,CVE-2018-18520 in elfutils,low,CVE-2018-18520
container_scanning,Clair,detected,CVE-2018-16869 in nettle,,CVE-2018-16869 in nettle,unknown,CVE-2018-16869
dependency_scanning,Gemnasium,detected,Regular Expression Denial of Service in debug,,Regular Expression Denial of Service in debug,unknown,yarn.lock:debug:gemnasium:37283ed4-0380-40d7-ada7-2d994afcc62a
dependency_scanning,Gemnasium,detected,Authentication bypass via incorrect DOM traversal and canonicalization in saml2-js,,Authentication bypass via incorrect DOM traversal and canonicalization in saml2-js,unknown,yarn.lock:saml2-js:gemnasium:9952e574-7b5b-46fa-a270-aeb694198a98
sast,Find Security Bugs,detected,Predictable pseudorandom number generator,,Predictable pseudorandom number generator,medium,818bf5dacb291e15d9e6dc3c5ac32178:PREDICTABLE_RANDOM:src/main/java/com/gitlab/security_products/tests/App.java:47
sast,Find Security Bugs,detected,Cipher with no integrity,,Cipher with no integrity,medium,e6449b89335daf53c0db4c0219bc1634:CIPHER_INTEGRITY:src/main/java/com/gitlab/security_products/tests/App.java:29
sast,Find Security Bugs,detected,Predictable pseudorandom number generator,,Predictable pseudorandom number generator,medium,e8ff1d01f74cd372f78da8f5247d3e73:PREDICTABLE_RANDOM:src/main/java/com/gitlab/security_products/tests/App.java:41
sast,Find Security Bugs,confirmed,ECB mode is insecure 2,,ECB mode is insecure,medium,ea0f905fc76f2739d5f10a1fd1e37a10:ECB_MODE:src/main/java/com/gitlab/security_products/tests/App.java:29
```
Group Name,Project Name,Scanner Type,Scanner Name,Status,Vulnerability,Details,Additional Info,Severity,CVE
Gitlab.org,Defend,container_scanning,Clair,confirmed,CVE-2017-16997 in glibc,,CVE-2017-16997 in glibc,critical,CVE-2017-16997
Gitlab.org,Defend,container_scanning,Clair,detected,CVE-2017-18269 in glibc,,CVE-2017-18269 in glibc,critical,CVE-2017-18269
Gitlab.org,Defend,container_scanning,Clair,detected,CVE-2018-1000001 in glibc,,CVE-2018-1000001 in glibc,high,CVE-2018-1000001
Gitlab.org,Defend,container_scanning,Clair,detected,CVE-2016-10228 in glibc,,CVE-2016-10228 in glibc,medium,CVE-2016-10228
Gitlab.org,Defend,container_scanning,Clair,confirmed,CVE-2010-4052 in glibc,,CVE-2010-4052 in glibc,low,CVE-2010-4052
Gitlab.org,Defend,container_scanning,Clair,detected,CVE-2018-18520 in elfutils,,CVE-2018-18520 in elfutils,low,CVE-2018-18520
Gitlab.org,Defend,container_scanning,Clair,detected,CVE-2018-16869 in nettle,,CVE-2018-16869 in nettle,unknown,CVE-2018-16869
Gitlab.org,Defend,dependency_scanning,Gemnasium,detected,Regular Expression Denial of Service in debug,,Regular Expression Denial of Service in debug,unknown,yarn.lock:debug:gemnasium:37283ed4-0380-40d7-ada7-2d994afcc62a
Gitlab.org,Defend,dependency_scanning,Gemnasium,detected,Authentication bypass via incorrect DOM traversal and canonicalization in saml2-js,,Authentication bypass via incorrect DOM traversal and canonicalization in saml2-js,unknown,yarn.lock:saml2-js:gemnasium:9952e574-7b5b-46fa-a270-aeb694198a98
Gitlab.org,Defend,sast,Find Security Bugs,detected,Predictable pseudorandom number generator,,Predictable pseudorandom number generator,medium,818bf5dacb291e15d9e6dc3c5ac32178:PREDICTABLE_RANDOM:src/main/java/com/gitlab/security_products/tests/App.java:47
Gitlab.org,Defend,sast,Find Security Bugs,detected,Cipher with no integrity,,Cipher with no integrity,medium,e6449b89335daf53c0db4c0219bc1634:CIPHER_INTEGRITY:src/main/java/com/gitlab/security_products/tests/App.java:29
Gitlab.org,Defend,sast,Find Security Bugs,detected,Predictable pseudorandom number generator,,Predictable pseudorandom number generator,medium,e8ff1d01f74cd372f78da8f5247d3e73:PREDICTABLE_RANDOM:src/main/java/com/gitlab/security_products/tests/App.java:41
Gitlab.org,Defend,sast,Find Security Bugs,confirmed,ECB mode is insecure 2,,ECB mode is insecure,medium,ea0f905fc76f2739d5f10a1fd1e37a10:ECB_MODE:src/main/java/com/gitlab/security_products/tests/App.java:29
Gitlab.org,Defend,```
......@@ -258,6 +258,14 @@ module Vulnerabilities
metadata.dig('evidence', 'summary')
end
def message
metadata.dig('message')
end
def cve
metadata.dig('cve')
end
alias_method :==, :eql? # eql? is necessary in some cases like array intersection
def eql?(other)
......
......@@ -29,6 +29,8 @@ class Vulnerability < ApplicationRecord
belongs_to :dismissed_by, class_name: 'User'
belongs_to :confirmed_by, class_name: 'User'
has_one :group, through: :project
has_many :findings, class_name: 'Vulnerabilities::Occurrence', inverse_of: :vulnerability
has_many :issue_links, class_name: 'Vulnerabilities::IssueLink', inverse_of: :vulnerability
has_many :related_issues, through: :issue_links, source: :issue do
......@@ -65,8 +67,6 @@ class Vulnerability < ApplicationRecord
scope :with_states, -> (states) { where(state: states) }
scope :counts_by_severity, -> { group(:severity).count }
delegate :default_branch, to: :project, prefix: :project
def self.counts_by_day_and_severity(num_days_in_past, end_date = Date.current)
return [] unless Feature.enabled?(:vulnerability_history, default_enabled: true)
......@@ -94,7 +94,11 @@ class Vulnerability < ApplicationRecord
findings.first
end
delegate :scanner_name, :metadata, to: :finding, prefix: true, allow_nil: true
delegate :scanner_name, :metadata, :message, :cve,
to: :finding, prefix: true, allow_nil: true
delegate :default_branch, :name, to: :project, prefix: true, allow_nil: true
delegate :name, to: :group, prefix: true, allow_nil: true
def resource_parent
project
......
......@@ -3,6 +3,19 @@
module VulnerabilityExports
module Exporters
class CsvService
MAPPING = {
'Group Name' => 'group_name',
'Project Name' => 'project_name',
'Scanner Type' => 'report_type',
'Scanner Name' => 'finding_scanner_name',
'Status' => 'state',
'Vulnerability' => 'title',
'Details' => 'description',
'Additional Info' => 'finding_message',
'Severity' => 'severity',
'CVE' => 'finding_cve'
}.freeze
attr_reader :vulnerabilities
def initialize(vulnerabilities)
......@@ -16,20 +29,7 @@ module VulnerabilityExports
private
def csv_builder
@csv_builder ||= CsvBuilder.new(vulnerabilities, header_to_value_hash)
end
def header_to_value_hash
{
'Scanner Type' => 'report_type',
'Scanner Name' => 'finding_scanner_name',
'Status' => 'state',
'Vulnerability' => 'title',
'Details' => 'description',
'Additional Info' => -> (vulnerability) { vulnerability.finding_metadata&.fetch('message', nil) },
'Severity' => 'severity',
'CVE' => -> (vulnerability) { vulnerability.finding_metadata&.fetch('cve', nil) }
}
@csv_builder ||= CsvBuilder.new(vulnerabilities, MAPPING)
end
end
end
......
---
title: Add `Group Name` and `Project Name` into vulnerability export CSV file
merge_request: 30755
author:
type: added
Scanner Type,Scanner Name,Status,Vulnerability,Details,Additional Info,Severity,CVE
container_scanning,Clair,detected,CVE-2017-16997 in glibc,,CVE-2017-16997 in glibc,critical,CVE-2017-16997
container_scanning,Clair,detected,CVE-2017-18269 in glibc,,CVE-2017-18269 in glibc,critical,CVE-2017-18269
container_scanning,Clair,detected,CVE-2018-1000001 in glibc,,CVE-2018-1000001 in glibc,high,CVE-2018-1000001
container_scanning,Clair,detected,CVE-2016-10228 in glibc,,CVE-2016-10228 in glibc,medium,CVE-2016-10228
container_scanning,Clair,detected,CVE-2010-4052 in glibc,,CVE-2010-4052 in glibc,low,CVE-2010-4052
container_scanning,Clair,detected,CVE-2018-18520 in elfutils,,CVE-2018-18520 in elfutils,low,CVE-2018-18520
container_scanning,Clair,detected,CVE-2018-16869 in nettle,,CVE-2018-16869 in nettle,unknown,CVE-2018-16869
dependency_scanning,Gemnasium,detected,Regular Expression Denial of Service in debug,,Regular Expression Denial of Service in debug,unknown,yarn.lock:debug:gemnasium:37283ed4-0380-40d7-ada7-2d994afcc62a
dependency_scanning,Gemnasium,detected,Authentication bypass via incorrect DOM traversal and canonicalization in saml2-js,,Authentication bypass via incorrect DOM traversal and canonicalization in saml2-js,unknown,yarn.lock:saml2-js:gemnasium:9952e574-7b5b-46fa-a270-aeb694198a98
sast,Find Security Bugs,detected,Predictable pseudorandom number generator,,Predictable pseudorandom number generator,medium,818bf5dacb291e15d9e6dc3c5ac32178:PREDICTABLE_RANDOM:src/main/java/com/gitlab/security_products/tests/App.java:47
sast,Find Security Bugs,detected,Cipher with no integrity,,Cipher with no integrity,medium,e6449b89335daf53c0db4c0219bc1634:CIPHER_INTEGRITY:src/main/java/com/gitlab/security_products/tests/App.java:29
sast,Find Security Bugs,detected,Predictable pseudorandom number generator,,Predictable pseudorandom number generator,medium,e8ff1d01f74cd372f78da8f5247d3e73:PREDICTABLE_RANDOM:src/main/java/com/gitlab/security_products/tests/App.java:41
sast,Find Security Bugs,detected,ECB mode is insecure,,ECB mode is insecure,medium,ea0f905fc76f2739d5f10a1fd1e37a10:ECB_MODE:src/main/java/com/gitlab/security_products/tests/App.java:29
Group Name,Project Name,Scanner Type,Scanner Name,Status,Vulnerability,Details,Additional Info,Severity,CVE
Gitlab.org,Defend,container_scanning,Clair,detected,CVE-2017-16997 in glibc,,CVE-2017-16997 in glibc,critical,CVE-2017-16997
Gitlab.org,Defend,container_scanning,Clair,detected,CVE-2017-18269 in glibc,,CVE-2017-18269 in glibc,critical,CVE-2017-18269
Gitlab.org,Defend,container_scanning,Clair,detected,CVE-2018-1000001 in glibc,,CVE-2018-1000001 in glibc,high,CVE-2018-1000001
Gitlab.org,Defend,container_scanning,Clair,detected,CVE-2016-10228 in glibc,,CVE-2016-10228 in glibc,medium,CVE-2016-10228
Gitlab.org,Defend,container_scanning,Clair,detected,CVE-2010-4052 in glibc,,CVE-2010-4052 in glibc,low,CVE-2010-4052
Gitlab.org,Defend,container_scanning,Clair,detected,CVE-2018-18520 in elfutils,,CVE-2018-18520 in elfutils,low,CVE-2018-18520
Gitlab.org,Defend,container_scanning,Clair,detected,CVE-2018-16869 in nettle,,CVE-2018-16869 in nettle,unknown,CVE-2018-16869
Gitlab.org,Defend,dependency_scanning,Gemnasium,detected,Regular Expression Denial of Service in debug,,Regular Expression Denial of Service in debug,unknown,yarn.lock:debug:gemnasium:37283ed4-0380-40d7-ada7-2d994afcc62a
Gitlab.org,Defend,dependency_scanning,Gemnasium,detected,Authentication bypass via incorrect DOM traversal and canonicalization in saml2-js,,Authentication bypass via incorrect DOM traversal and canonicalization in saml2-js,unknown,yarn.lock:saml2-js:gemnasium:9952e574-7b5b-46fa-a270-aeb694198a98
Gitlab.org,Defend,sast,Find Security Bugs,detected,Predictable pseudorandom number generator,,Predictable pseudorandom number generator,medium,818bf5dacb291e15d9e6dc3c5ac32178:PREDICTABLE_RANDOM:src/main/java/com/gitlab/security_products/tests/App.java:47
Gitlab.org,Defend,sast,Find Security Bugs,detected,Cipher with no integrity,,Cipher with no integrity,medium,e6449b89335daf53c0db4c0219bc1634:CIPHER_INTEGRITY:src/main/java/com/gitlab/security_products/tests/App.java:29
Gitlab.org,Defend,sast,Find Security Bugs,detected,Predictable pseudorandom number generator,,Predictable pseudorandom number generator,medium,e8ff1d01f74cd372f78da8f5247d3e73:PREDICTABLE_RANDOM:src/main/java/com/gitlab/security_products/tests/App.java:41
Gitlab.org,Defend,sast,Find Security Bugs,detected,ECB mode is insecure,,ECB mode is insecure,medium,ea0f905fc76f2739d5f10a1fd1e37a10:ECB_MODE:src/main/java/com/gitlab/security_products/tests/App.java:29
......@@ -588,4 +588,22 @@ describe Vulnerabilities::Occurrence do
expect(occurrence.evidence).to be_nil
end
end
describe '#message' do
let(:occurrence) { build(:vulnerabilities_occurrence) }
let(:expected_message) { occurrence.metadata['message'] }
subject { occurrence.message }
it { is_expected.to eql(expected_message) }
end
describe '#cve' do
let(:occurrence) { build(:vulnerabilities_occurrence) }
let(:expected_cve) { occurrence.metadata['cve'] }
subject { occurrence.cve }
it { is_expected.to eql(expected_cve) }
end
end
......@@ -39,6 +39,8 @@ describe Vulnerability do
it { is_expected.to belong_to(:dismissed_by).class_name('User') }
it { is_expected.to belong_to(:confirmed_by).class_name('User') }
it { is_expected.to have_one(:group).through(:project) }
it { is_expected.to have_many(:findings).class_name('Vulnerabilities::Occurrence').dependent(false) }
it { is_expected.to have_many(:notes).dependent(:delete_all) }
it { is_expected.to have_many(:user_mentions).class_name('VulnerabilityUserMention') }
......@@ -256,23 +258,14 @@ describe Vulnerability do
end
end
describe '#finding_scanner_name' do
let_it_be(:project) { create(:project, :with_vulnerability) }
let_it_be(:vulnerability) { project.vulnerabilities.first }
let_it_be(:finding) { create(:vulnerabilities_occurrence, vulnerability: vulnerability) }
subject(:finding_scanner_name) { vulnerability.finding_scanner_name }
it { is_expected.to eq(finding.scanner_name) }
end
describe '#project_default_branch' do
let_it_be(:project) { create(:project, :repository, :with_vulnerability) }
let_it_be(:vulnerability) { project.vulnerabilities.first }
subject { vulnerability.project_default_branch }
it { is_expected.to eq("master") }
describe 'delegations' do
it { is_expected.to delegate_method(:scanner_name).to(:finding).with_prefix.allow_nil }
it { is_expected.to delegate_method(:metadata).to(:finding).with_prefix.allow_nil }
it { is_expected.to delegate_method(:message).to(:finding).with_prefix.allow_nil }
it { is_expected.to delegate_method(:cve).to(:finding).with_prefix.allow_nil }
it { is_expected.to delegate_method(:default_branch).to(:project).with_prefix.allow_nil }
it { is_expected.to delegate_method(:name).to(:project).with_prefix.allow_nil }
it { is_expected.to delegate_method(:name).to(:group).with_prefix.allow_nil }
end
describe '#resolved_on_default_branch' do
......
......@@ -3,9 +3,6 @@
require 'spec_helper'
describe VulnerabilityExports::Exporters::CsvService do
let_it_be(:project) { create(:project, :public) }
let_it_be(:vulnerability) { create(:vulnerability, :with_findings, project: project) }
let(:export_csv_service) { described_class.new(Vulnerability.all) }
subject(:csv) { CSV.parse(export_csv_service.generate, headers: true) }
......@@ -23,20 +20,45 @@ describe VulnerabilityExports::Exporters::CsvService do
end
it 'includes the columns required for import' do
expect(csv.headers).to include('Scanner Type', 'Scanner Name', 'Status', 'Vulnerability', 'Details',
'Additional Info', 'Severity', 'CVE')
expect(csv.headers).to contain_exactly('Group Name', 'Project Name', 'Scanner Type', 'Scanner Name', 'Status',
'Vulnerability', 'Details', 'Additional Info', 'Severity', 'CVE')
end
context 'when a project belongs to a group' do
let_it_be(:group) { create(:group) }
let_it_be(:project) { create(:project, :public, group: group) }
let_it_be(:vulnerability) { create(:vulnerability, :with_findings, project: project) }
it 'includes proper values for each column type', :aggregate_failures do
expect(csv[0]['Group Name']).to eq group.name
expect(csv[0]['Project Name']).to eq project.name
expect(csv[0]['Scanner Type']).to eq vulnerability.report_type
expect(csv[0]['Scanner Name']).to eq vulnerability.finding_scanner_name
expect(csv[0]['Status']).to eq vulnerability.state
expect(csv[0]['Vulnerability']).to eq vulnerability.title
expect(csv[0]['Details']).to eq vulnerability.description
expect(csv[0]['Additional Info']).to eq vulnerability.finding_message
expect(csv[0]['Severity']).to eq vulnerability.severity
expect(csv[0]['CVE']).to eq vulnerability.finding_cve
end
end
it 'includes proper values for each column type' do
aggregate_failures do
context 'when a project belongs to a user' do
let_it_be(:user) { create(:user) }
let_it_be(:project) { create(:project, :public, namespace: user.namespace ) }
let_it_be(:vulnerability) { create(:vulnerability, :with_findings, project: project) }
it 'includes proper values for each column except group name' do
expect(csv[0]['Group Name']).to be_nil
expect(csv[0]['Project Name']).to eq project.name
expect(csv[0]['Scanner Type']).to eq vulnerability.report_type
expect(csv[0]['Scanner Name']).to eq vulnerability.finding_scanner_name
expect(csv[0]['Status']).to eq vulnerability.state
expect(csv[0]['Vulnerability']).to eq vulnerability.title
expect(csv[0]['Details']).to eq vulnerability.description
expect(csv[0]['Additional Info']).to eq vulnerability.finding_metadata['message']
expect(csv[0]['Additional Info']).to eq vulnerability.finding_message
expect(csv[0]['Severity']).to eq vulnerability.severity
expect(csv[0]['CVE']).to eq vulnerability.finding_metadata['cve']
expect(csv[0]['CVE']).to eq vulnerability.finding_cve
end
end
end
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment