Commit 50297719 authored by Fabio Pitino's avatar Fabio Pitino Committed by Shinya Maeda

Port Ci::Bridge to Core

Move most of the model and specs
parent f503bfec
---
title: Port `trigger` keyword in CI config to Core
merge_request: 24191
author:
type: fixed
...@@ -51,7 +51,8 @@ outbound connections for upstream and downstream pipeline dependencies. ...@@ -51,7 +51,8 @@ outbound connections for upstream and downstream pipeline dependencies.
## Creating multi-project pipelines from `.gitlab-ci.yml` ## Creating multi-project pipelines from `.gitlab-ci.yml`
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/8997) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.8. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/8997) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.8.
> - [Moved](https://gitlab.com/gitlab-org/gitlab/issues/199224) to GitLab Core in 12.8.
### Triggering a downstream pipeline using a bridge job ### Triggering a downstream pipeline using a bridge job
...@@ -181,7 +182,8 @@ the ones defined in the upstream project will take precedence. ...@@ -181,7 +182,8 @@ the ones defined in the upstream project will take precedence.
### Mirroring status from triggered pipeline ### Mirroring status from triggered pipeline
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/11238) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.3. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/11238) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.3.
> - [Moved](https://gitlab.com/gitlab-org/gitlab/issues/199224) to GitLab Core in 12.8.
You can mirror the pipeline status from the triggered pipeline to the source You can mirror the pipeline status from the triggered pipeline to the source
bridge job by using `strategy: depend`. For example: bridge job by using `strategy: depend`. For example:
......
...@@ -45,7 +45,7 @@ the child pipeline configuration. ...@@ -45,7 +45,7 @@ the child pipeline configuration.
## Examples ## Examples
The simplest case is [triggering a child pipeline](yaml/README.md#trigger-premium) using a The simplest case is [triggering a child pipeline](yaml/README.md#trigger) using a
local YAML file to define the pipeline configuration. In this case, the parent pipeline will local YAML file to define the pipeline configuration. In this case, the parent pipeline will
trigger the child pipeline, and continue without waiting: trigger the child pipeline, and continue without waiting:
......
...@@ -113,7 +113,7 @@ The following table lists available parameters for jobs: ...@@ -113,7 +113,7 @@ The following table lists available parameters for jobs:
| [`retry`](#retry) | When and how many times a job can be auto-retried in case of a failure. | | [`retry`](#retry) | When and how many times a job can be auto-retried in case of a failure. |
| [`timeout`](#timeout) | Define a custom job-level timeout that takes precedence over the project-wide setting. | | [`timeout`](#timeout) | Define a custom job-level timeout that takes precedence over the project-wide setting. |
| [`parallel`](#parallel) | How many instances of a job should be run in parallel. | | [`parallel`](#parallel) | How many instances of a job should be run in parallel. |
| [`trigger`](#trigger-premium) | Defines a downstream pipeline trigger. | | [`trigger`](#trigger) | Defines a downstream pipeline trigger. |
| [`include`](#include) | Allows this job to include external YAML files. Also available: `include:local`, `include:file`, `include:template`, and `include:remote`. | | [`include`](#include) | Allows this job to include external YAML files. Also available: `include:local`, `include:file`, `include:template`, and `include:remote`. |
| [`extends`](#extends) | Configuration entries that this job is going to inherit from. | | [`extends`](#extends) | Configuration entries that this job is going to inherit from. |
| [`pages`](#pages) | Upload the result of a job to use with GitLab Pages. | | [`pages`](#pages) | Upload the result of a job to use with GitLab Pages. |
...@@ -2572,9 +2572,10 @@ Please be aware that semaphore_test_boosters reports usages statistics to the au ...@@ -2572,9 +2572,10 @@ Please be aware that semaphore_test_boosters reports usages statistics to the au
You can then navigate to the **Jobs** tab of a new pipeline build and see your RSpec You can then navigate to the **Jobs** tab of a new pipeline build and see your RSpec
job split into three separate jobs. job split into three separate jobs.
### `trigger` **(PREMIUM)** ### `trigger`
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/8997) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.8. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/8997) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.8.
> - [Moved](https://gitlab.com/gitlab-org/gitlab/issues/199224) to GitLab Core in 12.8.
`trigger` allows you to define downstream pipeline trigger. When a job created `trigger` allows you to define downstream pipeline trigger. When a job created
from `trigger` definition is started by GitLab, a downstream pipeline gets from `trigger` definition is started by GitLab, a downstream pipeline gets
...@@ -3892,7 +3893,7 @@ job_no_git_strategy: ...@@ -3892,7 +3893,7 @@ job_no_git_strategy:
Triggers can be used to force a rebuild of a specific branch, tag or commit, Triggers can be used to force a rebuild of a specific branch, tag or commit,
with an API call when a pipeline gets created using a trigger token. with an API call when a pipeline gets created using a trigger token.
Not to be confused with [`trigger`](#trigger-premium). Not to be confused with [`trigger`](#trigger).
[Read more in the triggers documentation.](../triggers/README.md) [Read more in the triggers documentation.](../triggers/README.md)
......
# frozen_string_literal: true
module EE
module Gitlab
module Ci
module Config
module Entry
##
# Entry that represents a CI/CD Bridge job that is responsible for
# defining a downstream project trigger.
#
class Bridge < ::Gitlab::Config::Entry::Node
include ::Gitlab::Config::Entry::Configurable
include ::Gitlab::Config::Entry::Attributable
include ::Gitlab::Config::Entry::Inheritable
ALLOWED_KEYS = %i[trigger stage allow_failure only except
when extends variables needs rules].freeze
validations do
validates :config, allowed_keys: ALLOWED_KEYS
validates :config, presence: true
validates :name, presence: true
validates :name, type: Symbol
validates :config, disallowed_keys: {
in: %i[only except when start_in],
message: 'key may not be used with `rules`'
},
if: :has_rules?
with_options allow_nil: true do
validates :when,
inclusion: { in: %w[on_success on_failure always],
message: 'should be on_success, on_failure or always' }
validates :extends, type: String
validates :rules, array_of_hashes: true
end
validate on: :composed do
unless trigger.present? || bridge_needs.present?
errors.add(:config, 'should contain either a trigger or a needs:pipeline')
end
end
validate on: :composed do
next unless bridge_needs.present?
next if bridge_needs.one?
errors.add(:config, 'should contain at most one bridge need')
end
end
entry :trigger, ::EE::Gitlab::Ci::Config::Entry::Trigger,
description: 'CI/CD Bridge downstream trigger definition.',
inherit: false
entry :needs, ::Gitlab::Ci::Config::Entry::Needs,
description: 'CI/CD Bridge needs dependency definition.',
inherit: false,
metadata: { allowed_needs: %i[bridge job] }
entry :stage, ::Gitlab::Ci::Config::Entry::Stage,
description: 'Pipeline stage this job will be executed into.',
inherit: false
entry :only, ::Gitlab::Ci::Config::Entry::Policy,
description: 'Refs policy this job will be executed for.',
default: ::Gitlab::Ci::Config::Entry::Policy::DEFAULT_ONLY,
inherit: false
entry :except, ::Gitlab::Ci::Config::Entry::Policy,
description: 'Refs policy this job will be executed for.',
inherit: false
entry :rules, ::Gitlab::Ci::Config::Entry::Rules,
description: 'List of evaluable Rules to determine job inclusion.',
inherit: false,
metadata: {
allowed_when: %w[on_success on_failure always never manual delayed].freeze
}
entry :variables, ::Gitlab::Ci::Config::Entry::Variables,
description: 'Environment variables available for this job.',
inherit: false
helpers(*ALLOWED_KEYS)
attributes(*ALLOWED_KEYS)
def self.matching?(name, config)
::Feature.enabled?(:cross_project_pipeline_triggers, default_enabled: true) &&
!name.to_s.start_with?('.') &&
config.is_a?(Hash) &&
(config.key?(:trigger) || config.key?(:needs))
end
def self.visible?
true
end
def compose!(deps = nil)
super do
has_workflow_rules = deps&.workflow&.has_rules?
# If workflow:rules: or rules: are used
# they are considered not compatible
# with `only/except` defaults
#
# Context: https://gitlab.com/gitlab-org/gitlab/merge_requests/21742
if has_rules? || has_workflow_rules
# Remove only/except defaults
# defaults are not considered as defined
@entries.delete(:only) unless only_defined?
@entries.delete(:except) unless except_defined?
end
end
end
def has_rules?
@config&.key?(:rules)
end
def name
@metadata[:name]
end
def value
{ name: name,
trigger: (trigger_value if trigger_defined?),
needs: (needs_value if needs_defined?),
ignore: !!allow_failure,
stage: stage_value,
when: when_value,
extends: extends_value,
variables: (variables_value if variables_defined?),
rules: (rules_value if has_rules?),
only: only_value,
except: except_value }.compact
end
def bridge_needs
needs_value[:bridge] if needs_value
end
private
def overwrite_entry(deps, key, current_entry)
deps.default[key] unless current_entry.specified?
end
end
end
end
end
end
end
# frozen_string_literal: true
module EE
module Gitlab
module Ci
module Config
module Entry
module Jobs
extend ActiveSupport::Concern
prepended do
EE_TYPES = const_get(:TYPES, false) + [::EE::Gitlab::Ci::Config::Entry::Bridge]
end
class_methods do
extend ::Gitlab::Utils::Override
override :all_types
def all_types
EE_TYPES
end
end
end
end
end
end
end
end
...@@ -9,10 +9,13 @@ module EE ...@@ -9,10 +9,13 @@ module EE
extend ActiveSupport::Concern extend ActiveSupport::Concern
prepended do prepended do
# When defining a bridge that subscribes to an upstream pipeline:
# needs:pipeline: other/project
strategy :BridgeHash, strategy :BridgeHash,
class: EE::Gitlab::Ci::Config::Entry::Need::BridgeHash, class: EE::Gitlab::Ci::Config::Entry::Need::BridgeHash,
if: -> (config) { config.is_a?(Hash) && !config.key?(:job) && !config.key?(:project) } if: -> (config) { config.is_a?(Hash) && !config.key?(:job) && !config.key?(:project) }
# When defining DAG dependency across project/ref
strategy :CrossDependency, strategy :CrossDependency,
class: EE::Gitlab::Ci::Config::Entry::Need::CrossDependency, class: EE::Gitlab::Ci::Config::Entry::Need::CrossDependency,
if: -> (config) { config.is_a?(Hash) && (config.key?(:project) || config.key?(:ref)) } if: -> (config) { config.is_a?(Hash) && (config.key?(:project) || config.key?(:ref)) }
......
# frozen_string_literal: true
module EE
module Gitlab
module Ci
module Config
module Entry
##
# Entry that represents a cross-project downstream trigger.
#
class Trigger < ::Gitlab::Config::Entry::Simplifiable
strategy :SimpleTrigger, if: -> (config) { config.is_a?(String) }
strategy :ComplexTrigger, if: -> (config) { config.is_a?(Hash) }
class SimpleTrigger < ::Gitlab::Config::Entry::Node
include ::Gitlab::Config::Entry::Validatable
validations { validates :config, presence: true }
def value
{ project: @config }
end
end
class ComplexTrigger < ::Gitlab::Config::Entry::Simplifiable
strategy :CrossProjectTrigger, if: -> (config) { !config.key?(:include) }
strategy :SameProjectTrigger, if: -> (config) do
::Feature.enabled?(:ci_parent_child_pipeline, default_enabled: true) &&
config.key?(:include)
end
class CrossProjectTrigger < ::Gitlab::Config::Entry::Node
include ::Gitlab::Config::Entry::Validatable
include ::Gitlab::Config::Entry::Attributable
ALLOWED_KEYS = %i[project branch strategy].freeze
attributes :project, :branch, :strategy
validations do
validates :config, presence: true
validates :config, allowed_keys: ALLOWED_KEYS
validates :project, presence: true
validates :branch, type: String, allow_nil: true
validates :strategy, type: String, inclusion: { in: %w[depend], message: 'should be depend' }, allow_nil: true
end
end
class SameProjectTrigger < ::Gitlab::Config::Entry::Node
include ::Gitlab::Config::Entry::Validatable
include ::Gitlab::Config::Entry::Attributable
include ::Gitlab::Config::Entry::Configurable
INCLUDE_MAX_SIZE = 3
ALLOWED_KEYS = %i[strategy include].freeze
attributes :strategy
validations do
validates :config, presence: true
validates :config, allowed_keys: ALLOWED_KEYS
validates :strategy, type: String, inclusion: { in: %w[depend], message: 'should be depend' }, allow_nil: true
end
entry :include, ::Gitlab::Ci::Config::Entry::Includes,
description: 'List of external YAML files to include.',
reserved: true,
metadata: { max_size: INCLUDE_MAX_SIZE }
def value
@config
end
end
class UnknownStrategy < ::Gitlab::Config::Entry::Node
def errors
if ::Feature.enabled?(:ci_parent_child_pipeline, default_enabled: true)
['config must specify either project or include']
else
['config must specify project']
end
end
end
end
class UnknownStrategy < ::Gitlab::Config::Entry::Node
def errors
["#{location} has to be either a string or a hash"]
end
end
end
end
end
end
end
end
...@@ -2,52 +2,12 @@ ...@@ -2,52 +2,12 @@
require 'spec_helper' require 'spec_helper'
describe EE::Gitlab::Ci::Config::Entry::Bridge do describe Gitlab::Ci::Config::Entry::Bridge do
subject { described_class.new(config, name: :my_bridge) } subject { described_class.new(config, name: :my_bridge) }
it_behaves_like 'with inheritable CI config' do
let(:inheritable_key) { 'default' }
let(:inheritable_class) { Gitlab::Ci::Config::Entry::Default }
# These are entries defined in Default
# that we know that we don't want to inherit
# as they do not have sense in context of Bridge
let(:ignored_inheritable_columns) do
%i[before_script after_script image services cache interruptible timeout
retry tags artifacts]
end
end
describe '.matching?' do describe '.matching?' do
subject { described_class.matching?(name, config) } subject { described_class.matching?(name, config) }
context 'when config is not a hash' do
let(:name) { :my_trigger }
let(:config) { 'string' }
it { is_expected.to be_falsey }
end
context 'when config is a regular job' do
let(:name) { :my_trigger }
let(:config) do
{ script: 'ls -al' }
end
it { is_expected.to be_falsey }
context 'with rules' do
let(:config) do
{
script: 'ls -al',
rules: [{ if: '$VAR == "value"', when: 'always' }]
}
end
it { is_expected.to be_falsey }
end
end
context 'when config is a bridge job' do context 'when config is a bridge job' do
let(:name) { :my_trigger } let(:name) { :my_trigger }
let(:config) do let(:config) do
...@@ -55,26 +15,6 @@ describe EE::Gitlab::Ci::Config::Entry::Bridge do ...@@ -55,26 +15,6 @@ describe EE::Gitlab::Ci::Config::Entry::Bridge do
end end
it { is_expected.to be_truthy } it { is_expected.to be_truthy }
context 'with rules' do
let(:config) do
{
trigger: 'other-project',
rules: [{ if: '$VAR == "value"', when: 'always' }]
}
end
it { is_expected.to be_truthy }
end
end
context 'when config is a hidden job' do
let(:name) { '.my_trigger' }
let(:config) do
{ trigger: 'other-project' }
end
it { is_expected.to be_falsey }
end end
end end
...@@ -93,24 +33,6 @@ describe EE::Gitlab::Ci::Config::Entry::Bridge do ...@@ -93,24 +33,6 @@ describe EE::Gitlab::Ci::Config::Entry::Bridge do
} }
end end
context 'when trigger config is a non-empty string' do
let(:config) { { trigger: 'some/project' } }
describe '#valid?' do
it { is_expected.to be_valid }
end
describe '#value' do
it 'is returns a bridge job configuration' do
expect(subject.value).to eq(name: :my_bridge,
trigger: { project: 'some/project' },
ignore: false,
stage: 'test',
only: { refs: %w[branches tags] })
end
end
end
context 'when needs pipeline config is a non-empty string' do context 'when needs pipeline config is a non-empty string' do
let(:config) { { needs: { pipeline: 'some/project' } } } let(:config) { { needs: { pipeline: 'some/project' } } }
...@@ -129,27 +51,6 @@ describe EE::Gitlab::Ci::Config::Entry::Bridge do ...@@ -129,27 +51,6 @@ describe EE::Gitlab::Ci::Config::Entry::Bridge do
end end
end end
context 'when bridge trigger is a hash' do
let(:config) do
{ trigger: { project: 'some/project', branch: 'feature' } }
end
describe '#valid?' do
it { is_expected.to be_valid }
end
describe '#value' do
it 'is returns a bridge job configuration hash' do
expect(subject.value).to eq(name: :my_bridge,
trigger: { project: 'some/project',
branch: 'feature' },
ignore: false,
stage: 'test',
only: { refs: %w[branches tags] })
end
end
end
context 'when bridge configuration contains trigger, needs, when, extends, stage, only, except, and variables' do context 'when bridge configuration contains trigger, needs, when, extends, stage, only, except, and variables' do
let(:config) do let(:config) do
base_config.merge({ base_config.merge({
...@@ -162,45 +63,6 @@ describe EE::Gitlab::Ci::Config::Entry::Bridge do ...@@ -162,45 +63,6 @@ describe EE::Gitlab::Ci::Config::Entry::Bridge do
it { is_expected.to be_valid } it { is_expected.to be_valid }
end end
context 'when bridge configuration uses rules' do
let(:config) { base_config.merge({ rules: [{ if: '$VAR == null', when: 'never' }] }) }
it { is_expected.to be_valid }
end
context 'when bridge configuration uses rules with job:when' do
let(:config) do
base_config.merge({
when: 'always',
rules: [{ if: '$VAR == null', when: 'never' }]
})
end
it { is_expected.not_to be_valid }
end
context 'when bridge configuration uses rules with only' do
let(:config) do
base_config.merge({
only: { variables: %w[$SOMEVARIABLE] },
rules: [{ if: '$VAR == null', when: 'never' }]
})
end
it { is_expected.not_to be_valid }
end
context 'when bridge configuration uses rules with except' do
let(:config) do
base_config.merge({
except: { refs: %w[feature] },
rules: [{ if: '$VAR == null', when: 'never' }]
})
end
it { is_expected.not_to be_valid }
end
context 'when trigger config is nil' do context 'when trigger config is nil' do
let(:config) { { trigger: nil } } let(:config) { { trigger: nil } }
...@@ -229,43 +91,6 @@ describe EE::Gitlab::Ci::Config::Entry::Bridge do ...@@ -229,43 +91,6 @@ describe EE::Gitlab::Ci::Config::Entry::Bridge do
end end
end end
context 'when bridge has only job needs' do
let(:config) do
{
needs: ['some_job']
}
end
describe '#valid?' do
it { is_expected.not_to be_valid }
end
end
context 'when bridge has only cross projects dependencies' do
let(:config) do
{
needs: [
{
project: 'some/project',
job: 'some/job',
ref: 'some/ref',
artifacts: true
}
]
}
end
describe '#valid?' do
it { is_expected.not_to be_valid }
end
describe '#errors' do
it 'returns an error about cross dependencies' do
expect(subject.errors).to include('needs config uses invalid types: cross_dependency')
end
end
end
context 'when bridge has bridge and job needs' do context 'when bridge has bridge and job needs' do
let(:config) do let(:config) do
{ {
...@@ -324,35 +149,5 @@ describe EE::Gitlab::Ci::Config::Entry::Bridge do ...@@ -324,35 +149,5 @@ describe EE::Gitlab::Ci::Config::Entry::Bridge do
end end
end end
end end
context 'when bridge config contains unknown keys' do
let(:config) { { unknown: 123 } }
describe '#valid?' do
it { is_expected.not_to be_valid }
end
describe '#errors' do
it 'is returns an error about unknown config key' do
expect(subject.errors.first)
.to match /config contains unknown keys: unknown/
end
end
end
context 'when bridge config contains build-specific attributes' do
let(:config) { { script: 'something' } }
describe '#valid?' do
it { is_expected.not_to be_valid }
end
describe '#errors' do
it 'returns an error message' do
expect(subject.errors.first)
.to match /contains unknown keys: script/
end
end
end
end end
end end
# frozen_string_literal: true
require 'spec_helper'
describe Gitlab::Ci::Config::Entry::Jobs do
let(:config) do
{
'.hidden_job'.to_sym => { script: 'something' },
'.hidden_bridge'.to_sym => { trigger: 'my/project' },
regular_job: { script: 'something' },
my_trigger: { trigger: 'my/project' }
}
end
describe '.all_types' do
subject { described_class.all_types }
it { is_expected.to include(::EE::Gitlab::Ci::Config::Entry::Bridge) }
end
describe '.find_type' do
using RSpec::Parameterized::TableSyntax
subject { described_class.find_type(name, config[name]) }
context 'when cross-project pipeline triggers are enabled' do
before do
stub_feature_flags(cross_project_pipeline_triggers: true)
end
where(:name, :type) do
:'.hidden_job' | ::Gitlab::Ci::Config::Entry::Hidden
:'.hidden_bridge' | ::Gitlab::Ci::Config::Entry::Hidden
:regular_job | ::Gitlab::Ci::Config::Entry::Job
:my_trigger | ::EE::Gitlab::Ci::Config::Entry::Bridge
end
with_them do
it { is_expected.to eq(type) }
end
end
context 'when cross-project pipeline triggers are disabled' do
before do
stub_feature_flags(cross_project_pipeline_triggers: false)
end
where(:name, :type) do
:'.hidden_job' | ::Gitlab::Ci::Config::Entry::Hidden
:'.hidden_bridge' | ::Gitlab::Ci::Config::Entry::Hidden
:regular_job | ::Gitlab::Ci::Config::Entry::Job
:my_trigger | nil
end
with_them do
it { is_expected.to eq(type) }
end
end
end
describe '.new' do
subject do
described_class.new(config)
end
context 'when cross-project pipeline triggers are enabled' do
before do
stub_feature_flags(cross_project_pipeline_triggers: true)
subject.compose!
end
describe '#valid?' do
it { is_expected.to be_valid }
end
describe '#value' do
it 'returns a correct hash representing all jobs' do
expect(subject.value).to eq(
my_trigger: {
name: :my_trigger,
trigger: { project: 'my/project' },
stage: 'test',
only: { refs: %w[branches tags] },
ignore: false
},
regular_job: {
script: %w[something],
name: :regular_job,
stage: 'test',
only: { refs: %w[branches tags] },
variables: {},
ignore: false
})
end
end
end
context 'when cross-project pipeline triggers are disabled' do
before do
stub_feature_flags(cross_project_pipeline_triggers: false)
subject.compose!
end
describe '#valid?' do
it { is_expected.not_to be_valid }
end
end
end
end
...@@ -142,112 +142,6 @@ describe Ci::CreatePipelineService, '#execute' do ...@@ -142,112 +142,6 @@ describe Ci::CreatePipelineService, '#execute' do
end end
end end
describe 'child pipeline triggers' do
before do
stub_ci_pipeline_yaml_file <<~YAML
test:
script: rspec
deploy:
variables:
CROSS: downstream
stage: deploy
trigger:
include:
- local: path/to/child.yml
YAML
end
it 'creates bridge jobs correctly' do
pipeline = create_pipeline!
test = pipeline.statuses.find_by(name: 'test')
bridge = pipeline.statuses.find_by(name: 'deploy')
expect(pipeline).to be_persisted
expect(test).to be_a Ci::Build
expect(bridge).to be_a Ci::Bridge
expect(bridge.stage).to eq 'deploy'
expect(pipeline.statuses).to match_array [test, bridge]
expect(bridge.options).to eq(
'trigger' => { 'include' => [{ 'local' => 'path/to/child.yml' }] }
)
expect(bridge.yaml_variables)
.to include(key: 'CROSS', value: 'downstream', public: true)
end
end
describe 'child pipeline triggers' do
context 'when YAML is valid' do
before do
stub_ci_pipeline_yaml_file <<~YAML
test:
script: rspec
deploy:
variables:
CROSS: downstream
stage: deploy
trigger:
include:
- local: path/to/child.yml
YAML
end
it 'creates bridge jobs correctly' do
pipeline = create_pipeline!
test = pipeline.statuses.find_by(name: 'test')
bridge = pipeline.statuses.find_by(name: 'deploy')
expect(pipeline).to be_persisted
expect(test).to be_a Ci::Build
expect(bridge).to be_a Ci::Bridge
expect(bridge.stage).to eq 'deploy'
expect(pipeline.statuses).to match_array [test, bridge]
expect(bridge.options).to eq(
'trigger' => { 'include' => [{ 'local' => 'path/to/child.yml' }] }
)
expect(bridge.yaml_variables)
.to include(key: 'CROSS', value: 'downstream', public: true)
end
end
context 'when YAML is invalid' do
let(:config) do
{
test: { script: 'rspec' },
deploy: {
trigger: { include: included_files }
}
}
end
let(:included_files) do
Array.new(include_max_size + 1) do |index|
{ local: "file#{index}.yml" }
end
end
let(:include_max_size) do
EE::Gitlab::Ci::Config::Entry::Trigger::ComplexTrigger::SameProjectTrigger::INCLUDE_MAX_SIZE
end
before do
stub_ci_pipeline_yaml_file(YAML.dump(config))
end
it 'returns errors' do
pipeline = create_pipeline!
expect(pipeline.errors.full_messages.first).to match(/trigger:include config is too long/)
expect(pipeline.failure_reason).to eq 'config_error'
expect(pipeline).to be_persisted
expect(pipeline.status).to eq 'failed'
end
end
end
def create_pipeline! def create_pipeline!
service.execute(:push) service.execute(:push)
end end
......
# frozen_string_literal: true
module Gitlab
module Ci
class Config
module Entry
##
# Entry that represents a CI/CD Bridge job that is responsible for
# defining a downstream project trigger.
#
class Bridge < ::Gitlab::Config::Entry::Node
include ::Gitlab::Config::Entry::Configurable
include ::Gitlab::Config::Entry::Attributable
include ::Gitlab::Config::Entry::Inheritable
ALLOWED_KEYS = %i[trigger stage allow_failure only except
when extends variables needs rules].freeze
validations do
validates :config, allowed_keys: ALLOWED_KEYS
validates :config, presence: true
validates :name, presence: true
validates :name, type: Symbol
validates :config, disallowed_keys: {
in: %i[only except when start_in],
message: 'key may not be used with `rules`'
},
if: :has_rules?
with_options allow_nil: true do
validates :when,
inclusion: { in: %w[on_success on_failure always],
message: 'should be on_success, on_failure or always' }
validates :extends, type: String
validates :rules, array_of_hashes: true
end
validate on: :composed do
unless trigger.present? || bridge_needs.present?
errors.add(:config, 'should contain either a trigger or a needs:pipeline')
end
end
validate on: :composed do
next unless bridge_needs.present?
next if bridge_needs.one?
errors.add(:config, 'should contain at most one bridge need')
end
end
entry :trigger, ::Gitlab::Ci::Config::Entry::Trigger,
description: 'CI/CD Bridge downstream trigger definition.',
inherit: false
entry :needs, ::Gitlab::Ci::Config::Entry::Needs,
description: 'CI/CD Bridge needs dependency definition.',
inherit: false,
metadata: { allowed_needs: %i[job bridge] }
entry :stage, ::Gitlab::Ci::Config::Entry::Stage,
description: 'Pipeline stage this job will be executed into.',
inherit: false
entry :only, ::Gitlab::Ci::Config::Entry::Policy,
description: 'Refs policy this job will be executed for.',
default: ::Gitlab::Ci::Config::Entry::Policy::DEFAULT_ONLY,
inherit: false
entry :except, ::Gitlab::Ci::Config::Entry::Policy,
description: 'Refs policy this job will be executed for.',
inherit: false
entry :rules, ::Gitlab::Ci::Config::Entry::Rules,
description: 'List of evaluable Rules to determine job inclusion.',
inherit: false,
metadata: {
allowed_when: %w[on_success on_failure always never manual delayed].freeze
}
entry :variables, ::Gitlab::Ci::Config::Entry::Variables,
description: 'Environment variables available for this job.',
inherit: false
helpers(*ALLOWED_KEYS)
attributes(*ALLOWED_KEYS)
def self.matching?(name, config)
!name.to_s.start_with?('.') &&
config.is_a?(Hash) &&
(config.key?(:trigger) || config.key?(:needs))
end
def self.visible?
true
end
def compose!(deps = nil)
super do
has_workflow_rules = deps&.workflow&.has_rules?
# If workflow:rules: or rules: are used
# they are considered not compatible
# with `only/except` defaults
#
# Context: https://gitlab.com/gitlab-org/gitlab/merge_requests/21742
if has_rules? || has_workflow_rules
# Remove only/except defaults
# defaults are not considered as defined
@entries.delete(:only) unless only_defined?
@entries.delete(:except) unless except_defined?
end
end
end
def has_rules?
@config&.key?(:rules)
end
def name
@metadata[:name]
end
def value
{ name: name,
trigger: (trigger_value if trigger_defined?),
needs: (needs_value if needs_defined?),
ignore: !!allow_failure,
stage: stage_value,
when: when_value,
extends: extends_value,
variables: (variables_value if variables_defined?),
rules: (rules_value if has_rules?),
only: only_value,
except: except_value }.compact
end
def bridge_needs
needs_value[:bridge] if needs_value
end
private
def overwrite_entry(deps, key, current_entry)
deps.default[key] unless current_entry.specified?
end
end
end
end
end
end
...@@ -36,7 +36,7 @@ module Gitlab ...@@ -36,7 +36,7 @@ module Gitlab
end end
end end
TYPES = [Entry::Hidden, Entry::Job].freeze TYPES = [Entry::Hidden, Entry::Job, Entry::Bridge].freeze
private_constant :TYPES private_constant :TYPES
...@@ -77,5 +77,3 @@ module Gitlab ...@@ -77,5 +77,3 @@ module Gitlab
end end
end end
end end
::Gitlab::Ci::Config::Entry::Jobs.prepend_if_ee('::EE::Gitlab::Ci::Config::Entry::Jobs')
# frozen_string_literal: true
module Gitlab
module Ci
class Config
module Entry
##
# Entry that represents a cross-project downstream trigger.
#
class Trigger < ::Gitlab::Config::Entry::Simplifiable
strategy :SimpleTrigger, if: -> (config) { config.is_a?(String) }
strategy :ComplexTrigger, if: -> (config) { config.is_a?(Hash) }
class SimpleTrigger < ::Gitlab::Config::Entry::Node
include ::Gitlab::Config::Entry::Validatable
validations { validates :config, presence: true }
def value
{ project: @config }
end
end
class ComplexTrigger < ::Gitlab::Config::Entry::Simplifiable
strategy :CrossProjectTrigger, if: -> (config) { !config.key?(:include) }
strategy :SameProjectTrigger, if: -> (config) do
::Feature.enabled?(:ci_parent_child_pipeline, default_enabled: true) &&
config.key?(:include)
end
class CrossProjectTrigger < ::Gitlab::Config::Entry::Node
include ::Gitlab::Config::Entry::Validatable
include ::Gitlab::Config::Entry::Attributable
ALLOWED_KEYS = %i[project branch strategy].freeze
attributes :project, :branch, :strategy
validations do
validates :config, presence: true
validates :config, allowed_keys: ALLOWED_KEYS
validates :project, presence: true
validates :branch, type: String, allow_nil: true
validates :strategy, type: String, inclusion: { in: %w[depend], message: 'should be depend' }, allow_nil: true
end
end
class SameProjectTrigger < ::Gitlab::Config::Entry::Node
include ::Gitlab::Config::Entry::Validatable
include ::Gitlab::Config::Entry::Attributable
include ::Gitlab::Config::Entry::Configurable
INCLUDE_MAX_SIZE = 3
ALLOWED_KEYS = %i[strategy include].freeze
attributes :strategy
validations do
validates :config, presence: true
validates :config, allowed_keys: ALLOWED_KEYS
validates :strategy, type: String, inclusion: { in: %w[depend], message: 'should be depend' }, allow_nil: true
end
entry :include, ::Gitlab::Ci::Config::Entry::Includes,
description: 'List of external YAML files to include.',
reserved: true,
metadata: { max_size: INCLUDE_MAX_SIZE }
def value
@config
end
end
class UnknownStrategy < ::Gitlab::Config::Entry::Node
def errors
if ::Feature.enabled?(:ci_parent_child_pipeline, default_enabled: true)
['config must specify either project or include']
else
['config must specify project']
end
end
end
end
class UnknownStrategy < ::Gitlab::Config::Entry::Node
def errors
["#{location} has to be either a string or a hash"]
end
end
end
end
end
end
end
# frozen_string_literal: true
require 'spec_helper'
describe Gitlab::Ci::Config::Entry::Bridge do
subject { described_class.new(config, name: :my_bridge) }
it_behaves_like 'with inheritable CI config' do
let(:inheritable_key) { 'default' }
let(:inheritable_class) { Gitlab::Ci::Config::Entry::Default }
# These are entries defined in Default
# that we know that we don't want to inherit
# as they do not have sense in context of Bridge
let(:ignored_inheritable_columns) do
%i[before_script after_script image services cache interruptible timeout
retry tags artifacts]
end
end
describe '.matching?' do
subject { described_class.matching?(name, config) }
context 'when config is not a hash' do
let(:name) { :my_trigger }
let(:config) { 'string' }
it { is_expected.to be_falsey }
end
context 'when config is a regular job' do
let(:name) { :my_trigger }
let(:config) do
{ script: 'ls -al' }
end
it { is_expected.to be_falsey }
context 'with rules' do
let(:config) do
{
script: 'ls -al',
rules: [{ if: '$VAR == "value"', when: 'always' }]
}
end
it { is_expected.to be_falsey }
end
end
context 'when config is a bridge job' do
let(:name) { :my_trigger }
let(:config) do
{ trigger: 'other-project' }
end
it { is_expected.to be_truthy }
context 'with rules' do
let(:config) do
{
trigger: 'other-project',
rules: [{ if: '$VAR == "value"', when: 'always' }]
}
end
it { is_expected.to be_truthy }
end
end
context 'when config is a hidden job' do
let(:name) { '.my_trigger' }
let(:config) do
{ trigger: 'other-project' }
end
it { is_expected.to be_falsey }
end
end
describe '.new' do
before do
subject.compose!
end
let(:base_config) do
{
trigger: { project: 'some/project', branch: 'feature' },
extends: '.some-key',
stage: 'deploy',
variables: { VARIABLE: '123' }
}
end
context 'when trigger config is a non-empty string' do
let(:config) { { trigger: 'some/project' } }
describe '#valid?' do
it { is_expected.to be_valid }
end
describe '#value' do
it 'is returns a bridge job configuration' do
expect(subject.value).to eq(name: :my_bridge,
trigger: { project: 'some/project' },
ignore: false,
stage: 'test',
only: { refs: %w[branches tags] })
end
end
end
context 'when bridge trigger is a hash' do
let(:config) do
{ trigger: { project: 'some/project', branch: 'feature' } }
end
describe '#valid?' do
it { is_expected.to be_valid }
end
describe '#value' do
it 'is returns a bridge job configuration hash' do
expect(subject.value).to eq(name: :my_bridge,
trigger: { project: 'some/project',
branch: 'feature' },
ignore: false,
stage: 'test',
only: { refs: %w[branches tags] })
end
end
end
context 'when bridge configuration contains trigger, when, extends, stage, only, except, and variables' do
let(:config) do
base_config.merge({
when: 'always',
only: { variables: %w[$SOMEVARIABLE] },
except: { refs: %w[feature] }
})
end
it { is_expected.to be_valid }
end
context 'when bridge configuration uses rules' do
let(:config) { base_config.merge({ rules: [{ if: '$VAR == null', when: 'never' }] }) }
it { is_expected.to be_valid }
end
context 'when bridge configuration uses rules with job:when' do
let(:config) do
base_config.merge({
when: 'always',
rules: [{ if: '$VAR == null', when: 'never' }]
})
end
it { is_expected.not_to be_valid }
end
context 'when bridge configuration uses rules with only' do
let(:config) do
base_config.merge({
only: { variables: %w[$SOMEVARIABLE] },
rules: [{ if: '$VAR == null', when: 'never' }]
})
end
it { is_expected.not_to be_valid }
end
context 'when bridge configuration uses rules with except' do
let(:config) do
base_config.merge({
except: { refs: %w[feature] },
rules: [{ if: '$VAR == null', when: 'never' }]
})
end
it { is_expected.not_to be_valid }
end
context 'when bridge has only job needs' do
let(:config) do
{
needs: ['some_job']
}
end
describe '#valid?' do
it { is_expected.not_to be_valid }
end
end
context 'when bridge config contains unknown keys' do
let(:config) { { unknown: 123 } }
describe '#valid?' do
it { is_expected.not_to be_valid }
end
describe '#errors' do
it 'is returns an error about unknown config key' do
expect(subject.errors.first)
.to match /config contains unknown keys: unknown/
end
end
end
context 'when bridge config contains build-specific attributes' do
let(:config) { { script: 'something' } }
describe '#valid?' do
it { is_expected.not_to be_valid }
end
describe '#errors' do
it 'returns an error message' do
expect(subject.errors.first)
.to match /contains unknown keys: script/
end
end
end
end
end
...@@ -5,27 +5,31 @@ require 'spec_helper' ...@@ -5,27 +5,31 @@ require 'spec_helper'
describe Gitlab::Ci::Config::Entry::Jobs do describe Gitlab::Ci::Config::Entry::Jobs do
let(:entry) { described_class.new(config) } let(:entry) { described_class.new(config) }
let(:config) do
{
'.hidden_job'.to_sym => { script: 'something' },
'.hidden_bridge'.to_sym => { trigger: 'my/project' },
regular_job: { script: 'something' },
my_trigger: { trigger: 'my/project' }
}
end
describe '.all_types' do describe '.all_types' do
subject { described_class.all_types } subject { described_class.all_types }
it { is_expected.to include(::Gitlab::Ci::Config::Entry::Hidden) } it { is_expected.to include(::Gitlab::Ci::Config::Entry::Hidden) }
it { is_expected.to include(::Gitlab::Ci::Config::Entry::Job) } it { is_expected.to include(::Gitlab::Ci::Config::Entry::Job) }
it { is_expected.to include(::Gitlab::Ci::Config::Entry::Bridge) }
end end
describe '.find_type' do describe '.find_type' do
using RSpec::Parameterized::TableSyntax using RSpec::Parameterized::TableSyntax
let(:config) do
{
'.hidden_job'.to_sym => { script: 'something' },
regular_job: { script: 'something' },
invalid_job: 'text'
}
end
where(:name, :type) do where(:name, :type) do
:'.hidden_job' | ::Gitlab::Ci::Config::Entry::Hidden :'.hidden_job' | ::Gitlab::Ci::Config::Entry::Hidden
:'.hidden_bridge' | ::Gitlab::Ci::Config::Entry::Hidden
:regular_job | ::Gitlab::Ci::Config::Entry::Job :regular_job | ::Gitlab::Ci::Config::Entry::Job
:my_trigger | ::Gitlab::Ci::Config::Entry::Bridge
:invalid_job | nil :invalid_job | nil
end end
...@@ -42,8 +46,6 @@ describe Gitlab::Ci::Config::Entry::Jobs do ...@@ -42,8 +46,6 @@ describe Gitlab::Ci::Config::Entry::Jobs do
end end
context 'when entry config value is correct' do context 'when entry config value is correct' do
let(:config) { { rspec: { script: 'rspec' } } }
describe '#valid?' do describe '#valid?' do
it 'is valid' do it 'is valid' do
expect(entry).to be_valid expect(entry).to be_valid
...@@ -88,43 +90,41 @@ describe Gitlab::Ci::Config::Entry::Jobs do ...@@ -88,43 +90,41 @@ describe Gitlab::Ci::Config::Entry::Jobs do
entry.compose! entry.compose!
end end
let(:config) do
{ rspec: { script: 'rspec' },
spinach: { script: 'spinach' },
'.hidden'.to_sym => {} }
end
describe '#value' do describe '#value' do
it 'returns key value' do it 'returns key value' do
expect(entry.value).to eq( expect(entry.value).to eq(
rspec: { name: :rspec, my_trigger: {
script: %w[rspec],
ignore: false, ignore: false,
stage: 'test', name: :my_trigger,
only: { refs: %w[branches tags] }, only: { refs: %w[branches tags] },
variables: {} },
spinach: { name: :spinach,
script: %w[spinach],
ignore: false,
stage: 'test', stage: 'test',
trigger: { project: 'my/project' }
},
regular_job: {
ignore: false,
name: :regular_job,
only: { refs: %w[branches tags] }, only: { refs: %w[branches tags] },
variables: {} }) script: ['something'],
stage: 'test',
variables: {}
})
end end
end end
describe '#descendants' do describe '#descendants' do
it 'creates valid descendant nodes' do it 'creates valid descendant nodes' do
expect(entry.descendants.count).to eq 3 expect(entry.descendants.map(&:class)).to eq [
expect(entry.descendants.first(2)) Gitlab::Ci::Config::Entry::Hidden,
.to all(be_an_instance_of(Gitlab::Ci::Config::Entry::Job)) Gitlab::Ci::Config::Entry::Hidden,
expect(entry.descendants.last) Gitlab::Ci::Config::Entry::Job,
.to be_an_instance_of(Gitlab::Ci::Config::Entry::Hidden) Gitlab::Ci::Config::Entry::Bridge
]
end end
end end
describe '#value' do describe '#value' do
it 'returns value of visible jobs only' do it 'returns value of visible jobs only' do
expect(entry.value.keys).to eq [:rspec, :spinach] expect(entry.value.keys).to eq [:regular_job, :my_trigger]
end end
end end
end end
......
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
require 'spec_helper' require 'spec_helper'
describe EE::Gitlab::Ci::Config::Entry::Trigger do describe Gitlab::Ci::Config::Entry::Trigger do
subject { described_class.new(config) } subject { described_class.new(config) }
context 'when trigger config is a non-empty string' do context 'when trigger config is a non-empty string' do
......
...@@ -63,6 +63,26 @@ describe Ci::Bridge do ...@@ -63,6 +63,26 @@ describe Ci::Bridge do
end end
end end
describe 'state machine transitions' do
context 'when bridge points towards downstream' do
it 'schedules downstream pipeline creation' do
expect(bridge).to receive(:schedule_downstream_pipeline!)
bridge.enqueue!
end
end
end
describe 'state machine transitions' do
context 'when bridge points towards downstream' do
it 'schedules downstream pipeline creation' do
expect(bridge).to receive(:schedule_downstream_pipeline!)
bridge.enqueue!
end
end
end
describe '#inherit_status_from_downstream!' do describe '#inherit_status_from_downstream!' do
let(:downstream_pipeline) { build(:ci_pipeline, status: downstream_status) } let(:downstream_pipeline) { build(:ci_pipeline, status: downstream_status) }
......
...@@ -342,9 +342,7 @@ describe Ci::CreateCrossProjectPipelineService, '#execute' do ...@@ -342,9 +342,7 @@ describe Ci::CreateCrossProjectPipelineService, '#execute' do
let(:service) { described_class.new(upstream_project, upstream_project.owner) } let(:service) { described_class.new(upstream_project, upstream_project.owner) }
context 'that include the bridge job' do context 'that include the bridge job' do
# TODO: this is skipped because `trigger` keyword does not exist yet. it 'creates the downstream pipeline' do
# enabling it in the next MR: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/24393
xit 'creates the downstream pipeline' do
expect { service.execute(bridge) } expect { service.execute(bridge) }
.to change(downstream_project.ci_pipelines, :count).by(1) .to change(downstream_project.ci_pipelines, :count).by(1)
end end
......
# frozen_string_literal: true
require 'spec_helper'
describe Ci::CreatePipelineService, '#execute' do
set(:project) { create(:project, :repository) }
set(:user) { create(:user) }
let(:ref_name) { 'master' }
let(:service) do
params = { ref: ref_name,
before: '00000000',
after: project.commit.id,
commits: [{ message: 'some commit' }] }
described_class.new(project, user, params)
end
before do
project.add_developer(user)
stub_ci_pipeline_to_return_yaml_file
end
describe 'child pipeline triggers' do
before do
stub_ci_pipeline_yaml_file <<~YAML
test:
script: rspec
deploy:
variables:
CROSS: downstream
stage: deploy
trigger:
include:
- local: path/to/child.yml
YAML
end
it 'creates bridge jobs correctly' do
pipeline = create_pipeline!
test = pipeline.statuses.find_by(name: 'test')
bridge = pipeline.statuses.find_by(name: 'deploy')
expect(pipeline).to be_persisted
expect(test).to be_a Ci::Build
expect(bridge).to be_a Ci::Bridge
expect(bridge.stage).to eq 'deploy'
expect(pipeline.statuses).to match_array [test, bridge]
expect(bridge.options).to eq(
'trigger' => { 'include' => [{ 'local' => 'path/to/child.yml' }] }
)
expect(bridge.yaml_variables)
.to include(key: 'CROSS', value: 'downstream', public: true)
end
end
describe 'child pipeline triggers' do
context 'when YAML is valid' do
before do
stub_ci_pipeline_yaml_file <<~YAML
test:
script: rspec
deploy:
variables:
CROSS: downstream
stage: deploy
trigger:
include:
- local: path/to/child.yml
YAML
end
it 'creates bridge jobs correctly' do
pipeline = create_pipeline!
test = pipeline.statuses.find_by(name: 'test')
bridge = pipeline.statuses.find_by(name: 'deploy')
expect(pipeline).to be_persisted
expect(test).to be_a Ci::Build
expect(bridge).to be_a Ci::Bridge
expect(bridge.stage).to eq 'deploy'
expect(pipeline.statuses).to match_array [test, bridge]
expect(bridge.options).to eq(
'trigger' => { 'include' => [{ 'local' => 'path/to/child.yml' }] }
)
expect(bridge.yaml_variables)
.to include(key: 'CROSS', value: 'downstream', public: true)
end
end
context 'when YAML is invalid' do
let(:config) do
{
test: { script: 'rspec' },
deploy: {
trigger: { include: included_files }
}
}
end
let(:included_files) do
Array.new(include_max_size + 1) do |index|
{ local: "file#{index}.yml" }
end
end
let(:include_max_size) do
Gitlab::Ci::Config::Entry::Trigger::ComplexTrigger::SameProjectTrigger::INCLUDE_MAX_SIZE
end
before do
stub_ci_pipeline_yaml_file(YAML.dump(config))
end
it 'returns errors' do
pipeline = create_pipeline!
expect(pipeline.errors.full_messages.first).to match(/trigger:include config is too long/)
expect(pipeline.failure_reason).to eq 'config_error'
expect(pipeline).to be_persisted
expect(pipeline.status).to eq 'failed'
end
end
end
def create_pipeline!
service.execute(:push)
end
end
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment