Commit 8ad412e0 authored by Stan Hu's avatar Stan Hu Committed by Ruben Davila

Merge branch '21170-cycle-analytics' into 'master'

Cycle Analytics: first iteration

## What does this MR do?

- Implement the first iteration of the "Cycle Analytics" feature.

## What are the relevant issue numbers?

- Closes #21170 

## Screenshots

![cycle_analytics_screencast.gif](/uploads/d23c3c912caa6935fd47b53ca3a56b97/cycle_analytics.gif)

## Backend Tasks

- [x]  Implementation
    - [x]  Phases
        - [x]  Issue (Tracker)
        - [x]  Plan (Board)
        - [x]  Code (IDE)
        - [x]  Test (CI)
        - [x]  Review (MR)
        - [x]  Staging (CD)
        - [x]  Production (Total)
    - [x]  Make heuristics more modular
    - [x]  Scope to project
    - [x]  Date range (30 days, 90 days)
    - [x]  Access restriction
- [x]  Test
    - [x]  Find a better way to test these phases
    - [x]  Phases
        - [x]  Issue (Tracker)
        - [x]  Plan (Board)
        - [x]  Code (IDE)
        - [x]  Test (CI)
        - [x]  Review (MR)
        - [x]  Staging (CD)
        - [x]  Production (Total)
    - [x]  Test for "end case happens before start case"
    - [x]  Consolidate helper
- [x]  Miniboss review
- [x]  Performance testing with mock data
- [x]  Improve performance
    - [x]  Pre-calculate "merge requests closing issues
    - [x]  Pre-calculate everything else
- [x]  Test performance against 10k issues
- [x]  Test all pre-calculation code
    - [x]  Ci::Pipeline -> build start/finish
    - [x]  Ci::Pipeline#merge_requests
    - [x]  Issue -> record default metrics after save
    - [x]  MergeRequest -> record default metrics after save
    - [x]  Deployment -> Update "first_deployed_to_production_at" for MR metrics
    - [x]  Git Push -> Update "first commit mention" for issue metrics
    - [x]  Merge request create/update/refresh -> Update "merge requests closing issues"
- [x]  Remove `MergeRequestsClosingIssues` when necessary
- [x]  Changes to unblock Fatih
    - [x]  Add summary data
    - [x]  `stats` should be array
    - [x]  Let `stats` be `null` if all `stats` are null
- [x]  Indexes for "merge requests closing issues"
- [x]  Test summary data
- [x]  Scope everything to project
    - [x]  Find out why tests were passing
- [x]  Filter should include issues/MRs which have made it to production within the range
- [x]  Don't create duplicate `MergeRequestsClosingIssues`
- [x]  Fix tests
- [x]  MySQL median
- [x]  Assign to Douwe for review
- [x]  Fix conflicts
- [x]  Implement suggestions from Yorick's review
    - [x]  Test on PG
    - [x]  Test on MySQL
- [x]  Refactor
    - [x]  Cleanup
        - [x]  What happens if we have no data at all?
        - [x]  Extract common queries to methods / scopes
    - [x]  Remove unused queries
    - [x]  Downtime for foreign key migrations
    - [x]  Find a way around "if issue.metrics.present?" all over the place
    - [x]  Find a way around "if merge_request.metrics.present?" all over the place
    - [x]  Test migrations on a fresh database
        - [x]  MySQL
        - [x]  Pg
- [x]  Access issues
    - While the project is public and the visibility is set to "Everyone with access", you cannot visit the cycle analytics page when signed out.
- [x]  CHANGELOG
- [x]  Implement suggestions from Douwe's review
    - [x]  First set of comments
    - [x]  Second set of comments
    - [x]  Third set of comments
    - [x]  Fourth set of comments
- [x]  Make sure build is green
- [ ]  Make issue for "polish"
- [ ]  EE MR


See merge request !5986
parent 84e6b80b
...@@ -22,6 +22,7 @@ v 8.12.0 (unreleased) ...@@ -22,6 +22,7 @@ v 8.12.0 (unreleased)
- Pass the "Remember me" value to the U2F authentication form - Pass the "Remember me" value to the U2F authentication form
- Display stages in valid order in stages dropdown on build page - Display stages in valid order in stages dropdown on build page
- Only update projects.last_activity_at once per hour when creating a new event - Only update projects.last_activity_at once per hour when creating a new event
- Cycle analytics (first iteration) !5986
- Remove vendor prefixes for linear-gradient CSS (ClemMakesApps) - Remove vendor prefixes for linear-gradient CSS (ClemMakesApps)
- Move pushes_since_gc from the database to Redis - Move pushes_since_gc from the database to Redis
- Add font color contrast to external label in admin area (ClemMakesApps) - Add font color contrast to external label in admin area (ClemMakesApps)
......
...@@ -320,6 +320,7 @@ group :test do ...@@ -320,6 +320,7 @@ group :test do
gem 'webmock', '~> 1.21.0' gem 'webmock', '~> 1.21.0'
gem 'test_after_commit', '~> 0.4.2' gem 'test_after_commit', '~> 0.4.2'
gem 'sham_rack', '~> 1.3.6' gem 'sham_rack', '~> 1.3.6'
gem 'timecop', '~> 0.8.0'
end end
group :production do group :production do
......
...@@ -980,6 +980,7 @@ DEPENDENCIES ...@@ -980,6 +980,7 @@ DEPENDENCIES
teaspoon-jasmine (~> 2.2.0) teaspoon-jasmine (~> 2.2.0)
test_after_commit (~> 0.4.2) test_after_commit (~> 0.4.2)
thin (~> 1.7.0) thin (~> 1.7.0)
timecop (~> 0.8.0)
turbolinks (~> 2.5.0) turbolinks (~> 2.5.0)
u2f (~> 0.2.1) u2f (~> 0.2.1)
uglifier (~> 2.7.2) uglifier (~> 2.7.2)
......
((global) => {
const COOKIE_NAME = 'cycle_analytics_help_dismissed';
gl.CycleAnalytics = class CycleAnalytics {
constructor() {
const that = this;
this.isHelpDismissed = $.cookie(COOKIE_NAME);
this.vue = new Vue({
el: '#cycle-analytics',
name: 'CycleAnalytics',
created: this.fetchData(),
data: this.decorateData({ isLoading: true }),
methods: {
dismissLanding() {
that.dismissLanding();
}
}
});
}
fetchData(options) {
options = options || { startDate: 30 };
$.ajax({
url: $('#cycle-analytics').data('request-path'),
method: 'GET',
dataType: 'json',
contentType: 'application/json',
data: { start_date: options.startDate }
}).done((data) => {
this.vue.$data = this.decorateData(data);
this.initDropdown();
})
.error((data) => {
this.handleError(data);
})
.always(() => {
this.vue.isLoading = false;
})
}
decorateData(data) {
data.summary = data.summary || [];
data.stats = data.stats || [];
data.isHelpDismissed = this.isHelpDismissed;
data.isLoading = data.isLoading || false;
data.summary.forEach((item) => {
item.value = item.value || '-';
});
data.stats.forEach((item) => {
item.value = item.value || '- - -';
})
return data;
}
handleError(data) {
this.vue.$data = {
hasError: true,
isHelpDismissed: this.isHelpDismissed
};
new Flash('There was an error while fetching cycle analytics data.', 'alert');
}
dismissLanding() {
this.vue.isHelpDismissed = true;
$.cookie(COOKIE_NAME, true);
}
initDropdown() {
const $dropdown = $('.js-ca-dropdown');
const $label = $dropdown.find('.dropdown-label');
$dropdown.find('li a').off('click').on('click', (e) => {
e.preventDefault();
const $target = $(e.currentTarget);
const value = $target.data('value');
$label.text($target.text().trim());
this.vue.isLoading = true;
this.fetchData({ startDate: value });
})
}
}
})(window.gl || (window.gl = {}));
...@@ -94,6 +94,11 @@ ...@@ -94,6 +94,11 @@
break; break;
case "projects:merge_requests:conflicts": case "projects:merge_requests:conflicts":
window.mcui = new MergeConflictResolver() window.mcui = new MergeConflictResolver()
break;
case 'projects:merge_requests:index':
shortcut_handler = new ShortcutsNavigation();
Issuable.init();
break;
case 'dashboard:activity': case 'dashboard:activity':
new Activities(); new Activities();
break; break;
...@@ -185,6 +190,9 @@ ...@@ -185,6 +190,9 @@
new gl.ProtectedBranchCreate(); new gl.ProtectedBranchCreate();
new gl.ProtectedBranchEditList(); new gl.ProtectedBranchEditList();
break; break;
case 'projects:cycle_analytics:show':
new gl.CycleAnalytics();
break;
} }
switch (path.first()) { switch (path.first()) {
case 'admin': case 'admin':
......
#cycle-analytics {
margin: 24px auto 0;
width: 800px;
position: relative;
.panel {
.content-block {
padding: 24px 0;
border-bottom: none;
position: relative;
}
.column {
text-align: center;
.header {
font-size: 30px;
line-height: 38px;
font-weight: normal;
margin: 0;
}
.text {
color: $layout-link-gray;
margin: 0;
}
&:last-child {
text-align: right;
}
}
.dropdown {
position: relative;
top: 13px;
}
}
.bordered-box {
border: 1px solid $border-color;
@include border-radius($border-radius-default);
position: relative;
}
.content-list {
li {
padding: 18px $gl-padding $gl-padding;
.container-fluid {
padding: 0;
}
}
.title-col {
p {
margin: 0;
&.title {
line-height: 19px;
font-size: 15px;
font-weight: 600;
}
&:text {
color: #8c8c8c;
}
}
}
.value-col {
text-align: right;
span {
line-height: 42px;
}
}
}
.landing {
margin-bottom: $gl-padding;
overflow: hidden;
.dismiss-icon {
position: absolute;
right: $gl-padding;
cursor: pointer;
color: #b2b2b2;
}
svg {
margin: 0 20px;
float: left;
width: 136px;
height: 136px;
}
.inner-content {
width: 480px;
float: left;
h4 {
color: $gl-text-color;
font-size: 17px;
}
p {
color: #8c8c8c;
margin-bottom: $gl-padding;
}
}
}
.fa-spinner {
font-size: 28px;
position: relative;
margin-left: -20px;
left: 50%;
margin-top: 36px;
}
}
class Projects::CycleAnalyticsController < Projects::ApplicationController
include ActionView::Helpers::DateHelper
include ActionView::Helpers::TextHelper
before_action :authorize_read_cycle_analytics!
def show
@cycle_analytics = CycleAnalytics.new(@project, from: parse_start_date)
respond_to do |format|
format.html
format.json { render json: cycle_analytics_json }
end
end
private
def parse_start_date
case cycle_analytics_params[:start_date]
when '30' then 30.days.ago
when '90' then 90.days.ago
else 90.days.ago
end
end
def cycle_analytics_params
return {} unless params[:cycle_analytics].present?
{ start_date: params[:cycle_analytics][:start_date] }
end
def cycle_analytics_json
cycle_analytics_view_data = [[:issue, "Issue", "Time before an issue gets scheduled"],
[:plan, "Plan", "Time before an issue starts implementation"],
[:code, "Code", "Time until first merge request"],
[:test, "Test", "Total test time for all commits/merges"],
[:review, "Review", "Time between merge request creation and merge/close"],
[:staging, "Staging", "From merge request merge until deploy to production"],
[:production, "Production", "From issue creation until deploy to production"]]
stats = cycle_analytics_view_data.reduce([]) do |stats, (stage_method, stage_text, stage_description)|
value = @cycle_analytics.send(stage_method).presence
stats << {
title: stage_text,
description: stage_description,
value: value && !value.zero? ? distance_of_time_in_words(value) : nil
}
stats
end
issues = @cycle_analytics.summary.new_issues
commits = @cycle_analytics.summary.commits
deploys = @cycle_analytics.summary.deploys
summary = [
{ title: "New Issue".pluralize(issues), value: issues },
{ title: "Commit".pluralize(commits), value: commits },
{ title: "Deploy".pluralize(deploys), value: deploys }
]
{
summary: summary,
stats: stats
}
end
end
...@@ -46,6 +46,10 @@ module GitlabRoutingHelper ...@@ -46,6 +46,10 @@ module GitlabRoutingHelper
namespace_project_environments_path(project.namespace, project, *args) namespace_project_environments_path(project.namespace, project, *args)
end end
def project_cycle_analytics_path(project, *args)
namespace_project_cycle_analytics_path(project.namespace, project, *args)
end
def project_builds_path(project, *args) def project_builds_path(project, *args)
namespace_project_builds_path(project.namespace, project, *args) namespace_project_builds_path(project.namespace, project, *args)
end end
......
...@@ -56,6 +56,16 @@ module Ci ...@@ -56,6 +56,16 @@ module Ci
pipeline.finished_at = Time.now pipeline.finished_at = Time.now
end end
after_transition [:created, :pending] => :running do |pipeline|
MergeRequest::Metrics.where(merge_request_id: pipeline.merge_requests.map(&:id)).
update_all(latest_build_started_at: pipeline.started_at, latest_build_finished_at: nil)
end
after_transition any => [:success] do |pipeline|
MergeRequest::Metrics.where(merge_request_id: pipeline.merge_requests.map(&:id)).
update_all(latest_build_finished_at: pipeline.finished_at)
end
before_transition do |pipeline| before_transition do |pipeline|
pipeline.update_duration pipeline.update_duration
end end
...@@ -280,6 +290,16 @@ module Ci ...@@ -280,6 +290,16 @@ module Ci
project.execute_services(data, :pipeline_hooks) project.execute_services(data, :pipeline_hooks)
end end
# Merge requests for which the current pipeline is running against
# the merge request's latest commit.
def merge_requests
@merge_requests ||=
begin
project.merge_requests.where(source_branch: self.ref).
select { |merge_request| merge_request.pipeline.try(:id) == self.id }
end
end
private private
def pipeline_data def pipeline_data
......
...@@ -28,10 +28,13 @@ module Issuable ...@@ -28,10 +28,13 @@ module Issuable
loaded? && to_a.all? { |note| note.association(:award_emoji).loaded? } loaded? && to_a.all? { |note| note.association(:award_emoji).loaded? }
end end
end end
has_many :label_links, as: :target, dependent: :destroy has_many :label_links, as: :target, dependent: :destroy
has_many :labels, through: :label_links has_many :labels, through: :label_links
has_many :todos, as: :target, dependent: :destroy has_many :todos, as: :target, dependent: :destroy
has_one :metrics
validates :author, presence: true validates :author, presence: true
validates :title, presence: true, length: { within: 0..255 } validates :title, presence: true, length: { within: 0..255 }
...@@ -81,6 +84,7 @@ module Issuable ...@@ -81,6 +84,7 @@ module Issuable
acts_as_paranoid acts_as_paranoid
after_save :update_assignee_cache_counts, if: :assignee_id_changed? after_save :update_assignee_cache_counts, if: :assignee_id_changed?
after_save :record_metrics
def update_assignee_cache_counts def update_assignee_cache_counts
# make sure we flush the cache for both the old *and* new assignee # make sure we flush the cache for both the old *and* new assignee
...@@ -286,4 +290,9 @@ module Issuable ...@@ -286,4 +290,9 @@ module Issuable
def can_move?(*) def can_move?(*)
false false
end end
def record_metrics
metrics = self.metrics || create_metrics
metrics.record!
end
end end
class CycleAnalytics
include Gitlab::Database::Median
include Gitlab::Database::DateTime
def initialize(project, from:)
@project = project
@from = from
end
def summary
@summary ||= Summary.new(@project, from: @from)
end
def issue
calculate_metric(:issue,
Issue.arel_table[:created_at],
[Issue::Metrics.arel_table[:first_associated_with_milestone_at],
Issue::Metrics.arel_table[:first_added_to_board_at]])
end
def plan
calculate_metric(:plan,
[Issue::Metrics.arel_table[:first_associated_with_milestone_at],
Issue::Metrics.arel_table[:first_added_to_board_at]],
Issue::Metrics.arel_table[:first_mentioned_in_commit_at])
end
def code
calculate_metric(:code,
Issue::Metrics.arel_table[:first_mentioned_in_commit_at],
MergeRequest.arel_table[:created_at])
end
def test
calculate_metric(:test,
MergeRequest::Metrics.arel_table[:latest_build_started_at],
MergeRequest::Metrics.arel_table[:latest_build_finished_at])
end
def review
calculate_metric(:review,
MergeRequest.arel_table[:created_at],
MergeRequest::Metrics.arel_table[:merged_at])
end
def staging
calculate_metric(:staging,
MergeRequest::Metrics.arel_table[:merged_at],
MergeRequest::Metrics.arel_table[:first_deployed_to_production_at])
end
def production
calculate_metric(:production,
Issue.arel_table[:created_at],
MergeRequest::Metrics.arel_table[:first_deployed_to_production_at])
end
private
def calculate_metric(name, start_time_attrs, end_time_attrs)
cte_table = Arel::Table.new("cte_table_for_#{name}")
# Build a `SELECT` query. We find the first of the `end_time_attrs` that isn't `NULL` (call this end_time).
# Next, we find the first of the start_time_attrs that isn't `NULL` (call this start_time).
# We compute the (end_time - start_time) interval, and give it an alias based on the current
# cycle analytics stage.
interval_query = Arel::Nodes::As.new(
cte_table,
subtract_datetimes(base_query, end_time_attrs, start_time_attrs, name.to_s))
median_datetime(cte_table, interval_query, name)
end
# Join table with a row for every <issue,merge_request> pair (where the merge request
# closes the given issue) with issue and merge request metrics included. The metrics
# are loaded with an inner join, so issues / merge requests without metrics are
# automatically excluded.
def base_query
arel_table = MergeRequestsClosingIssues.arel_table
# Load issues
query = arel_table.join(Issue.arel_table).on(Issue.arel_table[:id].eq(arel_table[:issue_id])).
join(Issue::Metrics.arel_table).on(Issue.arel_table[:id].eq(Issue::Metrics.arel_table[:issue_id])).
where(Issue.arel_table[:project_id].eq(@project.id)).
where(Issue.arel_table[:deleted_at].eq(nil)).
where(Issue.arel_table[:created_at].gteq(@from))
# Load merge_requests
query = query.join(MergeRequest.arel_table, Arel::Nodes::OuterJoin).
on(MergeRequest.arel_table[:id].eq(arel_table[:merge_request_id])).
join(MergeRequest::Metrics.arel_table).
on(MergeRequest.arel_table[:id].eq(MergeRequest::Metrics.arel_table[:merge_request_id]))
# Limit to merge requests that have been deployed to production after `@from`
query.where(MergeRequest::Metrics.arel_table[:first_deployed_to_production_at].gteq(@from))
end
end
class CycleAnalytics
class Summary
def initialize(project, from:)
@project = project
@from = from
end
def new_issues
@project.issues.created_after(@from).count
end
def commits
repository = @project.repository.raw_repository
if @project.default_branch
repository.log(ref: @project.default_branch, after: @from).count
end
end
def deploys
@project.deployments.where("created_at > ?", @from).count
end
end
end
...@@ -42,4 +42,38 @@ class Deployment < ActiveRecord::Base ...@@ -42,4 +42,38 @@ class Deployment < ActiveRecord::Base
project.repository.is_ancestor?(commit.id, sha) project.repository.is_ancestor?(commit.id, sha)
end end
def update_merge_request_metrics!
return unless environment.update_merge_request_metrics?
merge_requests = project.merge_requests.
joins(:metrics).
where(target_branch: self.ref, merge_request_metrics: { first_deployed_to_production_at: nil }).
where("merge_request_metrics.merged_at <= ?", self.created_at)
if previous_deployment
merge_requests = merge_requests.where("merge_request_metrics.merged_at >= ?", previous_deployment.created_at)
end
# Need to use `map` instead of `select` because MySQL doesn't allow `SELECT`ing from the same table
# that we're updating.
merge_request_ids =
if Gitlab::Database.postgresql?
merge_requests.select(:id)
elsif Gitlab::Database.mysql?
merge_requests.map(&:id)
end
MergeRequest::Metrics.
where(merge_request_id: merge_request_ids, first_deployed_to_production_at: nil).
update_all(first_deployed_to_production_at: self.created_at)
end
def previous_deployment
@previous_deployment ||=
project.deployments.joins(:environment).
where(environments: { name: self.environment.name }, ref: self.ref).
where.not(id: self.id).
take
end
end end
...@@ -43,4 +43,8 @@ class Environment < ActiveRecord::Base ...@@ -43,4 +43,8 @@ class Environment < ActiveRecord::Base
last_deployment.includes_commit?(commit) last_deployment.includes_commit?(commit)
end end
def update_merge_request_metrics?
self.name == "production"
end
end end
...@@ -23,6 +23,8 @@ class Issue < ActiveRecord::Base ...@@ -23,6 +23,8 @@ class Issue < ActiveRecord::Base
has_many :events, as: :target, dependent: :destroy has_many :events, as: :target, dependent: :destroy
has_many :merge_requests_closing_issues, class_name: 'MergeRequestsClosingIssues', dependent: :delete_all
validates :project, presence: true validates :project, presence: true
scope :cared, ->(user) { where(assignee_id: user) } scope :cared, ->(user) { where(assignee_id: user) }
...@@ -36,6 +38,8 @@ class Issue < ActiveRecord::Base ...@@ -36,6 +38,8 @@ class Issue < ActiveRecord::Base
scope :order_due_date_asc, -> { reorder('issues.due_date IS NULL, issues.due_date ASC') } scope :order_due_date_asc, -> { reorder('issues.due_date IS NULL, issues.due_date ASC') }
scope :order_due_date_desc, -> { reorder('issues.due_date IS NULL, issues.due_date DESC') } scope :order_due_date_desc, -> { reorder('issues.due_date IS NULL, issues.due_date DESC') }
scope :created_after, -> (datetime) { where("created_at >= ?", datetime) }
attr_spammable :title, spam_title: true attr_spammable :title, spam_title: true
attr_spammable :description, spam_description: true attr_spammable :description, spam_description: true
......
class Issue::Metrics < ActiveRecord::Base
belongs_to :issue
def record!
if issue.milestone_id.present? && self.first_associated_with_milestone_at.blank?
self.first_associated_with_milestone_at = Time.now
end
if issue_assigned_to_list_label? && self.first_added_to_board_at.blank?
self.first_added_to_board_at = Time.now
end
self.save
end
private
def issue_assigned_to_list_label?
issue.labels.any? { |label| label.lists.present? }
end
end
...@@ -16,6 +16,8 @@ class MergeRequest < ActiveRecord::Base ...@@ -16,6 +16,8 @@ class MergeRequest < ActiveRecord::Base
has_many :events, as: :target, dependent: :destroy has_many :events, as: :target, dependent: :destroy
has_many :merge_requests_closing_issues, class_name: 'MergeRequestsClosingIssues', dependent: :delete_all
serialize :merge_params, Hash serialize :merge_params, Hash
after_create :ensure_merge_request_diff, unless: :importing? after_create :ensure_merge_request_diff, unless: :importing?
...@@ -501,6 +503,19 @@ class MergeRequest < ActiveRecord::Base ...@@ -501,6 +503,19 @@ class MergeRequest < ActiveRecord::Base
target_project target_project
end end
# If the merge request closes any issues, save this information in the
# `MergeRequestsClosingIssues` model. This is a performance optimization.
# Calculating this information for a number of merge requests requires
# running `ReferenceExtractor` on each of them separately.
def cache_merge_request_closes_issues!(current_user = self.author)
transaction do
self.merge_requests_closing_issues.delete_all
closes_issues(current_user).each do |issue|
self.merge_requests_closing_issues.create!(issue: issue)
end
end
end
def closes_issue?(issue) def closes_issue?(issue)
closes_issues.include?(issue) closes_issues.include?(issue)
end end
...@@ -508,7 +523,8 @@ class MergeRequest < ActiveRecord::Base ...@@ -508,7 +523,8 @@ class MergeRequest < ActiveRecord::Base
# Return the set of issues that will be closed if this merge request is accepted. # Return the set of issues that will be closed if this merge request is accepted.
def closes_issues(current_user = self.author) def closes_issues(current_user = self.author)
if target_branch == project.default_branch if target_branch == project.default_branch
messages = commits.map(&:safe_message) << description messages = [description]
messages.concat(commits.map(&:safe_message)) if merge_request_diff
Gitlab::ClosingIssueExtractor.new(project, current_user). Gitlab::ClosingIssueExtractor.new(project, current_user).
closed_by_message(messages.join("\n")) closed_by_message(messages.join("\n"))
......
class MergeRequest::Metrics < ActiveRecord::Base
belongs_to :merge_request
def record!
if merge_request.merged? && self.merged_at.blank?
self.merged_at = Time.now
end
self.save
end
end
class MergeRequestsClosingIssues < ActiveRecord::Base
belongs_to :merge_request
belongs_to :issue
validates :merge_request_id, uniqueness: { scope: :issue_id }, presence: true
validates :issue_id, presence: true
end
...@@ -46,6 +46,7 @@ class ProjectPolicy < BasePolicy ...@@ -46,6 +46,7 @@ class ProjectPolicy < BasePolicy
can! :create_issue can! :create_issue
can! :create_note can! :create_note
can! :upload_file can! :upload_file
can! :read_cycle_analytics
end end
def reporter_access! def reporter_access!
...@@ -204,6 +205,7 @@ class ProjectPolicy < BasePolicy ...@@ -204,6 +205,7 @@ class ProjectPolicy < BasePolicy
can! :read_commit_status can! :read_commit_status
can! :read_container_image can! :read_container_image
can! :download_code can! :download_code
can! :read_cycle_analytics
# NOTE: may be overridden by IssuePolicy # NOTE: may be overridden by IssuePolicy
can! :read_issue can! :read_issue
......
...@@ -4,7 +4,7 @@ class CreateDeploymentService < BaseService ...@@ -4,7 +4,7 @@ class CreateDeploymentService < BaseService
def execute(deployable = nil) def execute(deployable = nil)
environment = find_or_create_environment environment = find_or_create_environment
project.deployments.create( deployment = project.deployments.create(
environment: environment, environment: environment,
ref: params[:ref], ref: params[:ref],
tag: params[:tag], tag: params[:tag],
...@@ -12,6 +12,10 @@ class CreateDeploymentService < BaseService ...@@ -12,6 +12,10 @@ class CreateDeploymentService < BaseService
user: current_user, user: current_user,
deployable: deployable deployable: deployable
) )
deployment.update_merge_request_metrics!
deployment
end end
private private
......
...@@ -134,6 +134,7 @@ class GitPushService < BaseService ...@@ -134,6 +134,7 @@ class GitPushService < BaseService
end end
commit.create_cross_references!(authors[commit], closed_issues) commit.create_cross_references!(authors[commit], closed_issues)
update_issue_metrics(commit, authors)
end end
end end
...@@ -186,4 +187,11 @@ class GitPushService < BaseService ...@@ -186,4 +187,11 @@ class GitPushService < BaseService
def branch_name def branch_name
@branch_name ||= Gitlab::Git.ref_name(params[:ref]) @branch_name ||= Gitlab::Git.ref_name(params[:ref])
end end
def update_issue_metrics(commit, authors)
mentioned_issues = commit.all_references(authors[commit]).issues
Issue::Metrics.where(issue_id: mentioned_issues.map(&:id), first_mentioned_in_commit_at: nil).
update_all(first_mentioned_in_commit_at: commit.committed_date)
end
end end
...@@ -157,6 +157,10 @@ class IssuableBaseService < BaseService ...@@ -157,6 +157,10 @@ class IssuableBaseService < BaseService
# To be overridden by subclasses # To be overridden by subclasses
end end
def after_update(issuable)
# To be overridden by subclasses
end
def update_issuable(issuable, attributes) def update_issuable(issuable, attributes)
issuable.with_transaction_returning_status do issuable.with_transaction_returning_status do
issuable.update(attributes.merge(updated_by: current_user)) issuable.update(attributes.merge(updated_by: current_user))
...@@ -182,6 +186,7 @@ class IssuableBaseService < BaseService ...@@ -182,6 +186,7 @@ class IssuableBaseService < BaseService
end end
handle_changes(issuable, old_labels: old_labels, old_mentioned_users: old_mentioned_users) handle_changes(issuable, old_labels: old_labels, old_mentioned_users: old_mentioned_users)
after_update(issuable)
issuable.create_new_cross_references!(current_user) issuable.create_new_cross_references!(current_user)
execute_hooks(issuable, 'update') execute_hooks(issuable, 'update')
end end
......
...@@ -20,6 +20,7 @@ module MergeRequests ...@@ -20,6 +20,7 @@ module MergeRequests
event_service.open_mr(issuable, current_user) event_service.open_mr(issuable, current_user)
notification_service.new_merge_request(issuable, current_user) notification_service.new_merge_request(issuable, current_user)
todo_service.new_merge_request(issuable, current_user) todo_service.new_merge_request(issuable, current_user)
issuable.cache_merge_request_closes_issues!(current_user)
end end
end end
end end
...@@ -13,6 +13,7 @@ module MergeRequests ...@@ -13,6 +13,7 @@ module MergeRequests
reload_merge_requests reload_merge_requests
reset_merge_when_build_succeeds reset_merge_when_build_succeeds
mark_pending_todos_done mark_pending_todos_done
cache_merge_requests_closing_issues
# Leave a system note if a branch was deleted/added # Leave a system note if a branch was deleted/added
if branch_added? || branch_removed? if branch_added? || branch_removed?
...@@ -141,6 +142,14 @@ module MergeRequests ...@@ -141,6 +142,14 @@ module MergeRequests
end end
end end
# If the merge requests closes any issues, save this information in the
# `MergeRequestsClosingIssues` model (as a performance optimization).
def cache_merge_requests_closing_issues
@project.merge_requests.where(source_branch: @branch_name).each do |merge_request|
merge_request.cache_merge_request_closes_issues!(@current_user)
end
end
def filter_merge_requests(merge_requests) def filter_merge_requests(merge_requests)
merge_requests.uniq.select(&:source_project) merge_requests.uniq.select(&:source_project)
end end
......
...@@ -77,5 +77,9 @@ module MergeRequests ...@@ -77,5 +77,9 @@ module MergeRequests
def close_service def close_service
MergeRequests::CloseService MergeRequests::CloseService
end end
def after_update(issuable)
issuable.cache_merge_request_closes_issues!(current_user)
end
end end
end end
...@@ -47,7 +47,7 @@ ...@@ -47,7 +47,7 @@
Repository Repository
- if project_nav_tab? :pipelines - if project_nav_tab? :pipelines
= nav_link(controller: [:pipelines, :builds, :environments]) do = nav_link(controller: [:pipelines, :builds, :environments, :cycle_analytics]) do
= link_to project_pipelines_path(@project), title: 'Pipelines', class: 'shortcuts-pipelines' do = link_to project_pipelines_path(@project), title: 'Pipelines', class: 'shortcuts-pipelines' do
%span %span
Pipelines Pipelines
......
- @no_container = true
- page_title "Cycle Analytics"
= render "projects/pipelines/head"
#cycle-analytics{"v-cloak" => "true", data: { request_path: project_cycle_analytics_path(@project)}}
.bordered-box.landing.content-block{"v-if" => "!isHelpDismissed"}
= icon('times', class: 'dismiss-icon', "@click": "dismissLanding()")
= custom_icon('icon_cycle_analytics_splash')
.inner-content
%h4
Introducing Cycle Analytics
%p
Cycle Analytics gives an overview of how much time it takes to go from idea to production in your project.
= link_to "Read more", help_page_path('user/project/cycle_analytics'), target: '_blank', class: 'btn'
= icon("spinner spin", "v-show" => "isLoading")
.wrapper{"v-show" => "!isLoading && !hasError"}
.panel.panel-default
.panel-heading
Pipeline Health
.content-block
.container-fluid
.row
.col-xs-3.column{"v-for" => "item in summary"}
%h3.header {{item.value}}
%p.text {{item.title}}
.col-xs-3.column
.dropdown.inline.js-ca-dropdown
%button.dropdown-menu-toggle{"data-toggle" => "dropdown", :type => "button"}
%span.dropdown-label Last 30 days
%i.fa.fa-chevron-down
%ul.dropdown-menu.dropdown-menu-align-right
%li
%a{'href' => "#", 'data-value' => '30'}
Last 30 days
%li
%a{'href' => "#", 'data-value' => '90'}
Last 90 days
.bordered-box
%ul.content-list
%li{"v-for" => "item in stats"}
.container-fluid
.row
.col-xs-10.title-col
%p.title
{{item.title}}
%p.text
{{item.description}}
.col-xs-2.value-col
%span
{{item.value}}
...@@ -19,3 +19,9 @@ ...@@ -19,3 +19,9 @@
= link_to project_environments_path(@project), title: 'Environments', class: 'shortcuts-environments' do = link_to project_environments_path(@project), title: 'Environments', class: 'shortcuts-environments' do
%span %span
Environments Environments
- if can?(current_user, :read_cycle_analytics, @project)
= nav_link(controller: %w(cycle_analytics)) do
= link_to project_cycle_analytics_path(@project), title: 'Cycle Analytics' do
%span
Cycle Analytics
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 99 102" xmlns:xlink="http://www.w3.org/1999/xlink"><defs><path id="0" d="m35.12 56.988c4.083-4.385 5.968-12.155 5.968-24.04 0-20.2-15.874-32.16-15.874-32.16-1.114-.954-2.929-.979-4.04 0 0 0-15.874 11.957-15.874 32.16 0 11.882 1.884 19.652 5.968 24.04h23.848"/><mask id="1" width="35.783" height="56.924" x="0" y="0" fill="#fff"><use xlink:href="#0"/></mask></defs><g fill="none" fill-rule="evenodd" transform="translate(0-4)"><g transform="translate(32.15 3.976)"><g fill="#6b4fbb"><path d="m11.928 56.988l1.325-1.325v3.313c0 .737.59 1.325 1.325 1.325h17.229c.736 0 1.325-.59 1.325-1.325v-3.313l1.325 1.325h-22.53m22.53-1.325v3.313c0 1.464-1.18 2.651-2.651 2.651h-17.229c-1.464 0-2.651-1.178-2.651-2.651v-3.313h22.53m-5.964 7.361h.663c0 3.294-2.67 5.964-5.964 5.964-3.294 0-5.964-2.67-5.964-5.964h.663.663c0 2.562 2.077 4.639 4.639 4.639 2.562 0 4.639-2.077 4.639-4.639h.663"/><path d="m5.816 42.535c-.346-2.839-.515-6.03-.515-9.584 0-20.2 15.874-32.16 15.874-32.16 1.106-.979 2.921-.954 4.04 0 0 0 15.874 11.957 15.874 32.16 0 11.882-1.884 19.652-5.968 24.04h-23.848c-2.861-3.073-4.643-7.807-5.453-14.453-.06-.493-.115-.997-.164-1.511l-4.04 2.884c-.891.637-1.614 2.041-1.614 3.137v14.581c0 1.465.971 1.958 2.165 1.106l8.691-6.208c-.282-.332-.553-.681-.813-1.048l-8.648 6.177c-.147.105-.069.152-.069-.027v-14.581c0-.668.516-1.671 1.059-2.059l3.432-2.451m38.4 20.2c1.193.852 2.165.359 2.165-1.106v-14.581c0-1.096-.723-2.5-1.614-3.137l-4.04-2.884c-.049.514-.104 1.018-.164 1.511l3.432 2.451c.543.388 1.059 1.391 1.059 2.059v14.581c0 .179.078.132-.069.027l-8.648-6.177c-.26.367-.531.716-.813 1.048l8.691 6.208"/></g><use fill="#fff" stroke="#6b4fbb" stroke-width="2.651" mask="url(#1)" xlink:href="#0"/><g fill="#b5a7dd"><path d="m30.482 28.494c0-4.03-3.263-7.289-7.289-7.289-4.03 0-7.289 3.263-7.289 7.289 0 4.03 3.263 7.289 7.289 7.289 4.03 0 7.289-3.263 7.289-7.289m-15.904 0c0-4.758 3.857-8.614 8.614-8.614 4.758 0 8.614 3.857 8.614 8.614 0 4.758-3.857 8.614-8.614 8.614-4.758 0-8.614-3.857-8.614-8.614"/><path d="m27.17 28.494c0-2.196-1.78-3.976-3.976-3.976-2.196 0-3.976 1.78-3.976 3.976 0 2.196 1.78 3.976 3.976 3.976 2.196 0 3.976-1.78 3.976-3.976m-9.277 0c0-2.928 2.373-5.301 5.301-5.301 2.928 0 5.301 2.373 5.301 5.301 0 2.928-2.373 5.301-5.301 5.301-2.928 0-5.301-2.373-5.301-5.301"/></g><path fill="#6b4fbb" d="m34.458 87.47c0 1.098.89 1.988 1.988 1.988 1.098 0 1.988-.89 1.988-1.988 0-.366.297-.663.663-.663.366 0 .663.297.663.663 0 1.83-1.483 3.313-3.313 3.313-1.826 0-3.307-1.478-3.313-3.302 0-.002 0-.003 0-.005v-2.663c0-.363.294-.657.663-.657.366 0 .663.299.663.657v2.657m-21.2-6.615c0-.002 0-.003 0-.005v-2.663c0-.358-.297-.657-.663-.657-.369 0-.663.294-.663.657v2.657c0 1.098-.89 1.988-1.988 1.988-1.098 0-1.988-.89-1.988-1.988 0-.366-.297-.663-.663-.663-.366 0-.663.297-.663.663 0 1.83 1.483 3.313 3.313 3.313 1.826 0 3.307-1.477 3.313-3.302m5.301 7.285c0-.001 0-.002 0-.003v-16.576c0-.362-.297-.658-.663-.658-.369 0-.663.295-.663.658v16.571c0 2.01-1.632 3.645-3.645 3.645-2.01 0-3.645-1.632-3.645-3.645 0-.366-.297-.663-.663-.663-.366 0-.663.297-.663.663 0 2.745 2.225 4.97 4.97 4.97 2.742 0 4.966-2.221 4.97-4.963m10.602 8.607v-18.555c0-.365-.297-.661-.663-.661-.369 0-.663.296-.663.661v18.557c0 0 0 0 0 .001.001 2.744 2.226 4.968 4.97 4.968 2.745 0 4.97-2.225 4.97-4.97 0-.366-.297-.663-.663-.663-.366 0-.663.297-.663.663 0 2.01-1.632 3.645-3.645 3.645-2.01 0-3.645-1.632-3.645-3.645m3.976-25.19c0-.363.294-.657.663-.657.366 0 .663.299.663.657v2.663c0 .363-.294.657-.663.657-.366 0-.663-.299-.663-.657v-2.663m0 6.627c0-.363.294-.657.663-.657.366 0 .663.299.663.657v2.663c0 .363-.294.657-.663.657-.366 0-.663-.299-.663-.657v-2.663m-10.602-6.627c0-.363.294-.657.663-.657.366 0 .663.299.663.657v2.663c0 .363-.294.657-.663.657-.366 0-.663-.299-.663-.657v-2.663m5.301 0c0-.363.294-.657.663-.657.366 0 .663.299.663.657v2.663c0 .363-.294.657-.663.657-.366 0-.663-.299-.663-.657v-2.663m-5.301 6.627c0-.363.294-.657.663-.657.366 0 .663.299.663.657v2.663c0 .363-.294.657-.663.657-.366 0-.663-.299-.663-.657v-2.663m0 6.627c0-.363.294-.657.663-.657.366 0 .663.299.663.657v2.663c0 .363-.294.657-.663.657-.366 0-.663-.299-.663-.657v-2.663m-10.602-13.253c0-.363.294-.657.663-.657.366 0 .663.299.663.657v2.663c0 .363-.294.657-.663.657-.366 0-.663-.299-.663-.657v-2.663"/></g><path fill="#e2ddf2" d="m97.75 76.54c0-2.745-2.225-4.97-4.97-4.97-2.745 0-4.97 2.225-4.97 4.97 0 2.745 2.225 4.97 4.97 4.97 2.745 0 4.97-2.225 4.97-4.97m-8.614 0c0-2.01 1.632-3.645 3.645-3.645 2.01 0 3.645 1.632 3.645 3.645 0 2.01-1.632 3.645-3.645 3.645-2.01 0-3.645-1.632-3.645-3.645m-60.964-57.651c0-2.745-2.225-4.97-4.97-4.97-2.745 0-4.97 2.225-4.97 4.97 0 2.745 2.225 4.97 4.97 4.97 2.745 0 4.97-2.225 4.97-4.97m-8.614 0c0-2.01 1.632-3.645 3.645-3.645 2.01 0 3.645 1.632 3.645 3.645 0 2.01-1.632 3.645-3.645 3.645-2.01 0-3.645-1.632-3.645-3.645"/><path fill="#b5a7dd" d="m98.41 34.458c0-1.83-1.483-3.313-3.313-3.313-1.83 0-3.313 1.483-3.313 3.313 0 1.83 1.483 3.313 3.313 3.313 1.83 0 3.313-1.483 3.313-3.313m-5.301 0c0-1.098.89-1.988 1.988-1.988 1.098 0 1.988.89 1.988 1.988 0 1.098-.89 1.988-1.988 1.988-1.098 0-1.988-.89-1.988-1.988m-86.14 20.542c0-1.83-1.483-3.313-3.313-3.313-1.83 0-3.313 1.483-3.313 3.313 0 1.83 1.483 3.313 3.313 3.313 1.83 0 3.313-1.483 3.313-3.313m-5.301 0c0-1.098.89-1.988 1.988-1.988 1.098 0 1.988.89 1.988 1.988 0 1.098-.89 1.988-1.988 1.988-1.098 0-1.988-.89-1.988-1.988"/></g></svg>
...@@ -780,6 +780,8 @@ Rails.application.routes.draw do ...@@ -780,6 +780,8 @@ Rails.application.routes.draw do
resources :environments resources :environments
resource :cycle_analytics, only: [:show]
resources :builds, only: [:index, :show], constraints: { id: /\d+/ } do resources :builds, only: [:index, :show], constraints: { id: /\d+/ } do
collection do collection do
post :cancel_all post :cancel_all
......
require 'sidekiq/testing'
require './spec/support/test_env'
class Gitlab::Seeder::CycleAnalytics
def initialize(project, perf: false)
@project = project
@user = User.order(:id).last
@issue_count = perf ? 1000 : 5
stub_git_pre_receive!
end
# The GitLab API needn't be running for the fixtures to be
# created. Since we're performing a number of git actions
# here (like creating a branch or committing a file), we need
# to disable the `pre_receive` hook in order to remove this
# dependency on the GitLab API.
def stub_git_pre_receive!
GitHooksService.class_eval do
def run_hook(name)
[true, '']
end
end
end
def seed_metrics!
@issue_count.times do |index|
# Issue
Timecop.travel 5.days.from_now
title = "#{FFaker::Product.brand}-#{FFaker::Product.brand}-#{rand(1000)}"
issue = Issue.create(project: @project, title: title, author: @user)
issue_metrics = issue.metrics
# Milestones / Labels
Timecop.travel 5.days.from_now
if index.even?
issue_metrics.first_associated_with_milestone_at = rand(6..12).hours.from_now
else
issue_metrics.first_added_to_board_at = rand(6..12).hours.from_now
end
# Commit
Timecop.travel 5.days.from_now
issue_metrics.first_mentioned_in_commit_at = rand(6..12).hours.from_now
# MR
Timecop.travel 5.days.from_now
branch_name = "#{FFaker::Product.brand}-#{FFaker::Product.brand}-#{rand(1000)}"
@project.repository.add_branch(@user, branch_name, 'master')
merge_request = MergeRequest.create(target_project: @project, source_project: @project, source_branch: branch_name, target_branch: 'master', title: branch_name, author: @user)
merge_request_metrics = merge_request.metrics
# MR closing issues
Timecop.travel 5.days.from_now
MergeRequestsClosingIssues.create!(issue: issue, merge_request: merge_request)
# Merge
Timecop.travel 5.days.from_now
merge_request_metrics.merged_at = rand(6..12).hours.from_now
# Start build
Timecop.travel 5.days.from_now
merge_request_metrics.latest_build_started_at = rand(6..12).hours.from_now
# Finish build
Timecop.travel 5.days.from_now
merge_request_metrics.latest_build_finished_at = rand(6..12).hours.from_now
# Deploy to production
Timecop.travel 5.days.from_now
merge_request_metrics.first_deployed_to_production_at = rand(6..12).hours.from_now
issue_metrics.save!
merge_request_metrics.save!
print '.'
end
end
def seed!
Sidekiq::Testing.inline! do
issues = create_issues
puts '.'
# Stage 1
Timecop.travel 5.days.from_now
add_milestones_and_list_labels(issues)
print '.'
# Stage 2
Timecop.travel 5.days.from_now
branches = mention_in_commits(issues)
print '.'
# Stage 3
Timecop.travel 5.days.from_now
merge_requests = create_merge_requests_closing_issues(issues, branches)
print '.'
# Stage 4
Timecop.travel 5.days.from_now
run_builds(merge_requests)
print '.'
# Stage 5
Timecop.travel 5.days.from_now
merge_merge_requests(merge_requests)
print '.'
# Stage 6 / 7
Timecop.travel 5.days.from_now
deploy_to_production(merge_requests)
print '.'
end
print '.'
end
private
def create_issues
Array.new(@issue_count) do
issue_params = {
title: "Cycle Analytics: #{FFaker::Lorem.sentence(6)}",
description: FFaker::Lorem.sentence,
state: 'opened',
assignee: @project.team.users.sample
}
Issues::CreateService.new(@project, @project.team.users.sample, issue_params).execute
end
end
def add_milestones_and_list_labels(issues)
issues.shuffle.map.with_index do |issue, index|
Timecop.travel 12.hours.from_now
if index.even?
issue.update(milestone: @project.milestones.sample)
else
label_name = "#{FFaker::Product.brand}-#{FFaker::Product.brand}-#{rand(1000)}"
list_label = FactoryGirl.create(:label, title: label_name, project: issue.project)
FactoryGirl.create(:list, board: FactoryGirl.create(:board, project: issue.project), label: list_label)
issue.update(labels: [list_label])
end
issue
end
end
def mention_in_commits(issues)
issues.map do |issue|
Timecop.travel 12.hours.from_now
branch_name = filename = "#{FFaker::Product.brand}-#{FFaker::Product.brand}-#{rand(1000)}"
issue.project.repository.add_branch(@user, branch_name, 'master')
options = {
committer: issue.project.repository.user_to_committer(@user),
author: issue.project.repository.user_to_committer(@user),
commit: { message: "Commit for ##{issue.iid}", branch: branch_name, update_ref: true },
file: { content: "content", path: filename, update: false }
}
commit_sha = Gitlab::Git::Blob.commit(issue.project.repository, options)
issue.project.repository.commit(commit_sha)
GitPushService.new(issue.project,
@user,
oldrev: issue.project.repository.commit("master").sha,
newrev: commit_sha,
ref: 'refs/heads/master').execute
branch_name
end
end
def create_merge_requests_closing_issues(issues, branches)
issues.zip(branches).map do |issue, branch|
Timecop.travel 12.hours.from_now
opts = {
title: 'Cycle Analytics merge_request',
description: "Fixes #{issue.to_reference}",
source_branch: branch,
target_branch: 'master'
}
MergeRequests::CreateService.new(issue.project, @user, opts).execute
end
end
def run_builds(merge_requests)
merge_requests.each do |merge_request|
Timecop.travel 12.hours.from_now
service = Ci::CreatePipelineService.new(merge_request.project,
@user,
ref: "refs/heads/#{merge_request.source_branch}")
pipeline = service.execute(ignore_skip_ci: true, save_on_errors: false)
pipeline.run!
Timecop.travel rand(1..6).hours.from_now
pipeline.succeed!
end
end
def merge_merge_requests(merge_requests)
merge_requests.each do |merge_request|
Timecop.travel 12.hours.from_now
MergeRequests::MergeService.new(merge_request.project, @user).execute(merge_request)
end
end
def deploy_to_production(merge_requests)
merge_requests.each do |merge_request|
Timecop.travel 12.hours.from_now
CreateDeploymentService.new(merge_request.project, @user, {
environment: 'production',
ref: 'master',
tag: false,
sha: @project.repository.commit('master').sha
}).execute
end
end
end
Gitlab::Seeder.quiet do
if ENV['SEED_CYCLE_ANALYTICS']
Project.all.each do |project|
seeder = Gitlab::Seeder::CycleAnalytics.new(project)
seeder.seed!
end
elsif ENV['CYCLE_ANALYTICS_PERF_TEST']
seeder = Gitlab::Seeder::CycleAnalytics.new(Project.order(:id).first, perf: true)
seeder.seed!
elsif ENV['CYCLE_ANALYTICS_POPULATE_METRICS_DIRECTLY']
seeder = Gitlab::Seeder::CycleAnalytics.new(Project.order(:id).first, perf: true)
seeder.seed_metrics!
else
puts "Not running the cycle analytics seed file. Use the `SEED_CYCLE_ANALYTICS` environment variable to enable it."
end
end
# See http://doc.gitlab.com/ce/development/migration_style_guide.html
# for more information on how to write migrations for GitLab.
class AddTableIssueMetrics < ActiveRecord::Migration
include Gitlab::Database::MigrationHelpers
# Set this constant to true if this migration requires downtime.
DOWNTIME = true
# When a migration requires downtime you **must** uncomment the following
# constant and define a short and easy to understand explanation as to why the
# migration requires downtime.
DOWNTIME_REASON = 'Adding foreign key'
# When using the methods "add_concurrent_index" or "add_column_with_default"
# you must disable the use of transactions as these methods can not run in an
# existing transaction. When using "add_concurrent_index" make sure that this
# method is the _only_ method called in the migration, any other changes
# should go in a separate migration. This ensures that upon failure _only_ the
# index creation fails and can be retried or reverted easily.
#
# To disable transactions uncomment the following line and remove these
# comments:
# disable_ddl_transaction!
def change
create_table :issue_metrics do |t|
t.references :issue, index: { name: "index_issue_metrics" }, foreign_key: { on_delete: :cascade }, null: false
t.datetime 'first_mentioned_in_commit_at'
t.datetime 'first_associated_with_milestone_at'
t.datetime 'first_added_to_board_at'
t.timestamps null: false
end
end
end
# See http://doc.gitlab.com/ce/development/migration_style_guide.html
# for more information on how to write migrations for GitLab.
class AddTableMergeRequestMetrics < ActiveRecord::Migration
include Gitlab::Database::MigrationHelpers
# Set this constant to true if this migration requires downtime.
DOWNTIME = true
# When a migration requires downtime you **must** uncomment the following
# constant and define a short and easy to understand explanation as to why the
# migration requires downtime.
DOWNTIME_REASON = 'Adding foreign key'
# When using the methods "add_concurrent_index" or "add_column_with_default"
# you must disable the use of transactions as these methods can not run in an
# existing transaction. When using "add_concurrent_index" make sure that this
# method is the _only_ method called in the migration, any other changes
# should go in a separate migration. This ensures that upon failure _only_ the
# index creation fails and can be retried or reverted easily.
#
# To disable transactions uncomment the following line and remove these
# comments:
# disable_ddl_transaction!
def change
create_table :merge_request_metrics do |t|
t.references :merge_request, index: { name: "index_merge_request_metrics" }, foreign_key: { on_delete: :cascade }, null: false
t.datetime 'latest_build_started_at'
t.datetime 'latest_build_finished_at'
t.datetime 'first_deployed_to_production_at', index: true
t.datetime 'merged_at'
t.timestamps null: false
end
end
end
# See http://doc.gitlab.com/ce/development/migration_style_guide.html
# for more information on how to write migrations for GitLab.
class CreateMergeRequestsClosingIssues < ActiveRecord::Migration
include Gitlab::Database::MigrationHelpers
# Set this constant to true if this migration requires downtime.
DOWNTIME = true
# When a migration requires downtime you **must** uncomment the following
# constant and define a short and easy to understand explanation as to why the
# migration requires downtime.
DOWNTIME_REASON = 'Adding foreign keys'
# When using the methods "add_concurrent_index" or "add_column_with_default"
# you must disable the use of transactions as these methods can not run in an
# existing transaction. When using "add_concurrent_index" make sure that this
# method is the _only_ method called in the migration, any other changes
# should go in a separate migration. This ensures that upon failure _only_ the
# index creation fails and can be retried or reverted easily.
#
# To disable transactions uncomment the following line and remove these
# comments:
# disable_ddl_transaction!
def change
create_table :merge_requests_closing_issues do |t|
t.references :merge_request, foreign_key: { on_delete: :cascade }, index: true, null: false
t.references :issue, foreign_key: { on_delete: :cascade }, index: true, null: false
t.timestamps null: false
end
end
end
...@@ -11,7 +11,7 @@ ...@@ -11,7 +11,7 @@
# #
# It's strongly recommended that you check this file into your version control system. # It's strongly recommended that you check this file into your version control system.
ActiveRecord::Schema.define(version: 20160913212128) do ActiveRecord::Schema.define(version: 20160915081353) do
# These are extensions that must be enabled in order to support this database # These are extensions that must be enabled in order to support this database
enable_extension "plpgsql" enable_extension "plpgsql"
...@@ -439,6 +439,17 @@ ActiveRecord::Schema.define(version: 20160913212128) do ...@@ -439,6 +439,17 @@ ActiveRecord::Schema.define(version: 20160913212128) do
add_index "identities", ["user_id"], name: "index_identities_on_user_id", using: :btree add_index "identities", ["user_id"], name: "index_identities_on_user_id", using: :btree
create_table "issue_metrics", force: :cascade do |t|
t.integer "issue_id", null: false
t.datetime "first_associated_with_milestone_at"
t.datetime "first_added_to_board_at"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.datetime "first_mentioned_in_commit_at"
end
add_index "issue_metrics", ["issue_id"], name: "index_issue_metrics", using: :btree
create_table "issues", force: :cascade do |t| create_table "issues", force: :cascade do |t|
t.string "title" t.string "title"
t.integer "assignee_id" t.integer "assignee_id"
...@@ -581,6 +592,18 @@ ActiveRecord::Schema.define(version: 20160913212128) do ...@@ -581,6 +592,18 @@ ActiveRecord::Schema.define(version: 20160913212128) do
add_index "merge_request_diffs", ["merge_request_id"], name: "index_merge_request_diffs_on_merge_request_id", using: :btree add_index "merge_request_diffs", ["merge_request_id"], name: "index_merge_request_diffs_on_merge_request_id", using: :btree
create_table "merge_request_metrics", force: :cascade do |t|
t.integer "merge_request_id", null: false
t.datetime "latest_build_started_at"
t.datetime "latest_build_finished_at"
t.datetime "first_deployed_to_production_at"
t.datetime "merged_at"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
end
add_index "merge_request_metrics", ["merge_request_id"], name: "index_merge_request_metrics", using: :btree
create_table "merge_requests", force: :cascade do |t| create_table "merge_requests", force: :cascade do |t|
t.string "target_branch", null: false t.string "target_branch", null: false
t.string "source_branch", null: false t.string "source_branch", null: false
...@@ -622,6 +645,16 @@ ActiveRecord::Schema.define(version: 20160913212128) do ...@@ -622,6 +645,16 @@ ActiveRecord::Schema.define(version: 20160913212128) do
add_index "merge_requests", ["title"], name: "index_merge_requests_on_title", using: :btree add_index "merge_requests", ["title"], name: "index_merge_requests_on_title", using: :btree
add_index "merge_requests", ["title"], name: "index_merge_requests_on_title_trigram", using: :gin, opclasses: {"title"=>"gin_trgm_ops"} add_index "merge_requests", ["title"], name: "index_merge_requests_on_title_trigram", using: :gin, opclasses: {"title"=>"gin_trgm_ops"}
create_table "merge_requests_closing_issues", force: :cascade do |t|
t.integer "merge_request_id", null: false
t.integer "issue_id", null: false
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
end
add_index "merge_requests_closing_issues", ["issue_id"], name: "index_merge_requests_closing_issues_on_issue_id", using: :btree
add_index "merge_requests_closing_issues", ["merge_request_id"], name: "index_merge_requests_closing_issues_on_merge_request_id", using: :btree
create_table "milestones", force: :cascade do |t| create_table "milestones", force: :cascade do |t|
t.string "title", null: false t.string "title", null: false
t.integer "project_id", null: false t.integer "project_id", null: false
...@@ -1147,8 +1180,12 @@ ActiveRecord::Schema.define(version: 20160913212128) do ...@@ -1147,8 +1180,12 @@ ActiveRecord::Schema.define(version: 20160913212128) do
add_index "web_hooks", ["project_id"], name: "index_web_hooks_on_project_id", using: :btree add_index "web_hooks", ["project_id"], name: "index_web_hooks_on_project_id", using: :btree
add_foreign_key "boards", "projects" add_foreign_key "boards", "projects"
add_foreign_key "issue_metrics", "issues", on_delete: :cascade
add_foreign_key "lists", "boards" add_foreign_key "lists", "boards"
add_foreign_key "lists", "labels" add_foreign_key "lists", "labels"
add_foreign_key "merge_request_metrics", "merge_requests", on_delete: :cascade
add_foreign_key "merge_requests_closing_issues", "issues", on_delete: :cascade
add_foreign_key "merge_requests_closing_issues", "merge_requests", on_delete: :cascade
add_foreign_key "personal_access_tokens", "users" add_foreign_key "personal_access_tokens", "users"
add_foreign_key "protected_branch_merge_access_levels", "protected_branches" add_foreign_key "protected_branch_merge_access_levels", "protected_branches"
add_foreign_key "protected_branch_push_access_levels", "protected_branches" add_foreign_key "protected_branch_push_access_levels", "protected_branches"
......
module Gitlab
module Database
module DateTime
# Find the first of the `end_time_attrs` that isn't `NULL`. Subtract from it
# the first of the `start_time_attrs` that isn't NULL. `SELECT` the resulting interval
# along with an alias specified by the `as` parameter.
#
# Note: For MySQL, the interval is returned in seconds.
# For PostgreSQL, the interval is returned as an INTERVAL type.
def subtract_datetimes(query_so_far, end_time_attrs, start_time_attrs, as)
diff_fn = if Gitlab::Database.postgresql?
Arel::Nodes::Subtraction.new(
Arel::Nodes::NamedFunction.new("COALESCE", Array.wrap(end_time_attrs)),
Arel::Nodes::NamedFunction.new("COALESCE", Array.wrap(start_time_attrs)))
elsif Gitlab::Database.mysql?
Arel::Nodes::NamedFunction.new(
"TIMESTAMPDIFF",
[Arel.sql('second'),
Arel::Nodes::NamedFunction.new("COALESCE", Array.wrap(start_time_attrs)),
Arel::Nodes::NamedFunction.new("COALESCE", Array.wrap(end_time_attrs))])
end
query_so_far.project(diff_fn.as(as))
end
end
end
end
# https://www.periscopedata.com/blog/medians-in-sql.html
module Gitlab
module Database
module Median
def median_datetime(arel_table, query_so_far, column_sym)
median_queries =
if Gitlab::Database.postgresql?
pg_median_datetime_sql(arel_table, query_so_far, column_sym)
elsif Gitlab::Database.mysql?
mysql_median_datetime_sql(arel_table, query_so_far, column_sym)
end
results = Array.wrap(median_queries).map do |query|
ActiveRecord::Base.connection.execute(query)
end
extract_median(results).presence
end
def extract_median(results)
result = results.compact.first
if Gitlab::Database.postgresql?
result = result.first.presence
median = result['median'] if result
median.to_f if median
elsif Gitlab::Database.mysql?
result.to_a.flatten.first
end
end
def mysql_median_datetime_sql(arel_table, query_so_far, column_sym)
query = arel_table.
from(arel_table.project(Arel.sql('*')).order(arel_table[column_sym]).as(arel_table.table_name)).
project(average([arel_table[column_sym]], 'median')).
where(
Arel::Nodes::Between.new(
Arel.sql("(select @row_id := @row_id + 1)"),
Arel::Nodes::And.new(
[Arel.sql('@ct/2.0'),
Arel.sql('@ct/2.0 + 1')]
)
)
).
# Disallow negative values
where(arel_table[column_sym].gteq(0))
[
Arel.sql("CREATE TEMPORARY TABLE IF NOT EXISTS #{query_so_far.to_sql}"),
Arel.sql("set @ct := (select count(1) from #{arel_table.table_name});"),
Arel.sql("set @row_id := 0;"),
query.to_sql,
Arel.sql("DROP TEMPORARY TABLE IF EXISTS #{arel_table.table_name};")
]
end
def pg_median_datetime_sql(arel_table, query_so_far, column_sym)
# Create a CTE with the column we're operating on, row number (after sorting by the column
# we're operating on), and count of the table we're operating on (duplicated across) all rows
# of the CTE. For example, if we're looking to find the median of the `projects.star_count`
# column, the CTE might look like this:
#
# star_count | row_id | ct
# ------------+--------+----
# 5 | 1 | 3
# 9 | 2 | 3
# 15 | 3 | 3
cte_table = Arel::Table.new("ordered_records")
cte = Arel::Nodes::As.new(
cte_table,
arel_table.
project(
arel_table[column_sym].as(column_sym.to_s),
Arel::Nodes::Over.new(Arel::Nodes::NamedFunction.new("row_number", []),
Arel::Nodes::Window.new.order(arel_table[column_sym])).as('row_id'),
arel_table.project("COUNT(1)").as('ct')).
# Disallow negative values
where(arel_table[column_sym].gteq(zero_interval)))
# From the CTE, select either the middle row or the middle two rows (this is accomplished
# by 'where cte.row_id between cte.ct / 2.0 AND cte.ct / 2.0 + 1'). Find the average of the
# selected rows, and this is the median value.
cte_table.project(average([extract_epoch(cte_table[column_sym])], "median")).
where(
Arel::Nodes::Between.new(
cte_table[:row_id],
Arel::Nodes::And.new(
[(cte_table[:ct] / Arel.sql('2.0')),
(cte_table[:ct] / Arel.sql('2.0') + 1)]
)
)
).
with(query_so_far, cte).
to_sql
end
private
def average(args, as)
Arel::Nodes::NamedFunction.new("AVG", args, as)
end
def extract_epoch(arel_attribute)
Arel.sql(%Q{EXTRACT(EPOCH FROM "#{arel_attribute.relation.name}"."#{arel_attribute.name}")})
end
# Need to cast '0' to an INTERVAL before we can check if the interval is positive
def zero_interval
Arel::Nodes::NamedFunction.new("CAST", [Arel.sql("'0' AS INTERVAL")])
end
end
end
end
...@@ -3,11 +3,12 @@ FactoryGirl.define do ...@@ -3,11 +3,12 @@ FactoryGirl.define do
sha '97de212e80737a608d939f648d959671fb0a0142' sha '97de212e80737a608d939f648d959671fb0a0142'
ref 'master' ref 'master'
tag false tag false
project nil
environment factory: :environment environment factory: :environment
after(:build) do |deployment, evaluator| after(:build) do |deployment, evaluator|
deployment.project = deployment.environment.project deployment.project ||= deployment.environment.project
end end
end end
end end
...@@ -187,6 +187,37 @@ describe Ci::Pipeline, models: true do ...@@ -187,6 +187,37 @@ describe Ci::Pipeline, models: true do
end end
end end
describe "merge request metrics" do
let(:project) { FactoryGirl.create :project }
let(:pipeline) { FactoryGirl.create(:ci_empty_pipeline, status: 'created', project: project, ref: 'master', sha: project.repository.commit('master').id) }
let!(:merge_request) { create(:merge_request, source_project: project, source_branch: pipeline.ref) }
context 'when transitioning to running' do
it 'records the build start time' do
time = Time.now
Timecop.freeze(time) { build.run }
expect(merge_request.reload.metrics.latest_build_started_at).to be_within(1.second).of(time)
end
it 'clears the build end time' do
build.run
expect(merge_request.reload.metrics.latest_build_finished_at).to be_nil
end
end
context 'when transitioning to success' do
it 'records the build end time' do
build.run
time = Time.now
Timecop.freeze(time) { build.success }
expect(merge_request.reload.metrics.latest_build_finished_at).to be_within(1.second).of(time)
end
end
end
def create_build(name, queued_at = current, started_from = 0) def create_build(name, queued_at = current, started_from = 0)
create(:ci_build, create(:ci_build,
name: name, name: name,
...@@ -468,4 +499,28 @@ describe Ci::Pipeline, models: true do ...@@ -468,4 +499,28 @@ describe Ci::Pipeline, models: true do
stage_idx: stage_idx) stage_idx: stage_idx)
end end
end end
describe "#merge_requests" do
let(:project) { FactoryGirl.create :project }
let(:pipeline) { FactoryGirl.create(:ci_empty_pipeline, status: 'created', project: project, ref: 'master', sha: project.repository.commit('master').id) }
it "returns merge requests whose `diff_head_sha` matches the pipeline's SHA" do
merge_request = create(:merge_request, source_project: project, source_branch: pipeline.ref)
expect(pipeline.merge_requests).to eq([merge_request])
end
it "doesn't return merge requests whose source branch doesn't match the pipeline's ref" do
create(:merge_request, source_project: project, source_branch: 'feature', target_branch: 'master')
expect(pipeline.merge_requests).to be_empty
end
it "doesn't return merge requests whose `diff_head_sha` doesn't match the pipeline's SHA" do
create(:merge_request, source_project: project, source_branch: pipeline.ref)
allow_any_instance_of(MergeRequest).to receive(:diff_head_sha) { '97de212e80737a608d939f648d959671fb0a0142b' }
expect(pipeline.merge_requests).to be_empty
end
end
end end
require 'spec_helper'
describe 'CycleAnalytics#code', feature: true do
extend CycleAnalyticsHelpers::TestGeneration
let(:project) { create(:project) }
let(:from_date) { 10.days.ago }
let(:user) { create(:user, :admin) }
subject { CycleAnalytics.new(project, from: from_date) }
generate_cycle_analytics_spec(
phase: :code,
data_fn: -> (context) { { issue: context.create(:issue, project: context.project) } },
start_time_conditions: [["issue mentioned in a commit",
-> (context, data) do
context.create_commit_referencing_issue(data[:issue])
end]],
end_time_conditions: [["merge request that closes issue is created",
-> (context, data) do
context.create_merge_request_closing_issue(data[:issue])
end]],
post_fn: -> (context, data) do
context.merge_merge_requests_closing_issue(data[:issue])
context.deploy_master
end)
context "when a regular merge request (that doesn't close the issue) is created" do
it "returns nil" do
5.times do
issue = create(:issue, project: project)
create_commit_referencing_issue(issue)
create_merge_request_closing_issue(issue, message: "Closes nothing")
merge_merge_requests_closing_issue(issue)
deploy_master
end
expect(subject.code).to be_nil
end
end
end
require 'spec_helper'
describe 'CycleAnalytics#issue', models: true do
extend CycleAnalyticsHelpers::TestGeneration
let(:project) { create(:project) }
let(:from_date) { 10.days.ago }
let(:user) { create(:user, :admin) }
subject { CycleAnalytics.new(project, from: from_date) }
generate_cycle_analytics_spec(
phase: :issue,
data_fn: -> (context) { { issue: context.build(:issue, project: context.project) } },
start_time_conditions: [["issue created", -> (context, data) { data[:issue].save }]],
end_time_conditions: [["issue associated with a milestone",
-> (context, data) do
if data[:issue].persisted?
data[:issue].update(milestone: context.create(:milestone, project: context.project))
end
end],
["list label added to issue",
-> (context, data) do
if data[:issue].persisted?
data[:issue].update(label_ids: [context.create(:label, lists: [context.create(:list)]).id])
end
end]],
post_fn: -> (context, data) do
if data[:issue].persisted?
context.create_merge_request_closing_issue(data[:issue].reload)
context.merge_merge_requests_closing_issue(data[:issue])
context.deploy_master
end
end)
context "when a regular label (instead of a list label) is added to the issue" do
it "returns nil" do
5.times do
regular_label = create(:label)
issue = create(:issue, project: project)
issue.update(label_ids: [regular_label.id])
create_merge_request_closing_issue(issue)
merge_merge_requests_closing_issue(issue)
deploy_master
end
expect(subject.issue).to be_nil
end
end
end
require 'spec_helper'
describe 'CycleAnalytics#plan', feature: true do
extend CycleAnalyticsHelpers::TestGeneration
let(:project) { create(:project) }
let(:from_date) { 10.days.ago }
let(:user) { create(:user, :admin) }
subject { CycleAnalytics.new(project, from: from_date) }
generate_cycle_analytics_spec(
phase: :plan,
data_fn: -> (context) do
{
issue: context.create(:issue, project: context.project),
branch_name: context.random_git_name
}
end,
start_time_conditions: [["issue associated with a milestone",
-> (context, data) do
data[:issue].update(milestone: context.create(:milestone, project: context.project))
end],
["list label added to issue",
-> (context, data) do
data[:issue].update(label_ids: [context.create(:label, lists: [context.create(:list)]).id])
end]],
end_time_conditions: [["issue mentioned in a commit",
-> (context, data) do
context.create_commit_referencing_issue(data[:issue], branch_name: data[:branch_name])
end]],
post_fn: -> (context, data) do
context.create_merge_request_closing_issue(data[:issue], source_branch: data[:branch_name])
context.merge_merge_requests_closing_issue(data[:issue])
context.deploy_master
end)
context "when a regular label (instead of a list label) is added to the issue" do
it "returns nil" do
branch_name = random_git_name
label = create(:label)
issue = create(:issue, project: project)
issue.update(label_ids: [label.id])
create_commit_referencing_issue(issue, branch_name: branch_name)
create_merge_request_closing_issue(issue, source_branch: branch_name)
merge_merge_requests_closing_issue(issue)
deploy_master
expect(subject.issue).to be_nil
end
end
end
require 'spec_helper'
describe 'CycleAnalytics#production', feature: true do
extend CycleAnalyticsHelpers::TestGeneration
let(:project) { create(:project) }
let(:from_date) { 10.days.ago }
let(:user) { create(:user, :admin) }
subject { CycleAnalytics.new(project, from: from_date) }
generate_cycle_analytics_spec(
phase: :production,
data_fn: -> (context) { { issue: context.build(:issue, project: context.project) } },
start_time_conditions: [["issue is created", -> (context, data) { data[:issue].save }]],
before_end_fn: lambda do |context, data|
context.create_merge_request_closing_issue(data[:issue])
context.merge_merge_requests_closing_issue(data[:issue])
end,
end_time_conditions:
[["merge request that closes issue is deployed to production", -> (context, data) { context.deploy_master }],
["production deploy happens after merge request is merged (along with other changes)",
lambda do |context, data|
# Make other changes on master
sha = context.project.repository.commit_file(context.user, context.random_git_name, "content", "commit message", 'master', false)
context.project.repository.commit(sha)
context.deploy_master
end]])
context "when a regular merge request (that doesn't close the issue) is merged and deployed" do
it "returns nil" do
5.times do
merge_request = create(:merge_request)
MergeRequests::MergeService.new(project, user).execute(merge_request)
deploy_master
end
expect(subject.production).to be_nil
end
end
context "when the deployment happens to a non-production environment" do
it "returns nil" do
5.times do
issue = create(:issue, project: project)
merge_request = create_merge_request_closing_issue(issue)
MergeRequests::MergeService.new(project, user).execute(merge_request)
deploy_master(environment: 'staging')
end
expect(subject.production).to be_nil
end
end
end
require 'spec_helper'
describe 'CycleAnalytics#review', feature: true do
extend CycleAnalyticsHelpers::TestGeneration
let(:project) { create(:project) }
let(:from_date) { 10.days.ago }
let(:user) { create(:user, :admin) }
subject { CycleAnalytics.new(project, from: from_date) }
generate_cycle_analytics_spec(
phase: :review,
data_fn: -> (context) { { issue: context.create(:issue, project: context.project) } },
start_time_conditions: [["merge request that closes issue is created",
-> (context, data) do
context.create_merge_request_closing_issue(data[:issue])
end]],
end_time_conditions: [["merge request that closes issue is merged",
-> (context, data) do
context.merge_merge_requests_closing_issue(data[:issue])
end]],
post_fn: -> (context, data) { context.deploy_master })
context "when a regular merge request (that doesn't close the issue) is created and merged" do
it "returns nil" do
5.times do
MergeRequests::MergeService.new(project, user).execute(create(:merge_request))
deploy_master
end
expect(subject.review).to be_nil
end
end
end
require 'spec_helper'
describe 'CycleAnalytics#staging', feature: true do
extend CycleAnalyticsHelpers::TestGeneration
let(:project) { create(:project) }
let(:from_date) { 10.days.ago }
let(:user) { create(:user, :admin) }
subject { CycleAnalytics.new(project, from: from_date) }
generate_cycle_analytics_spec(
phase: :staging,
data_fn: lambda do |context|
issue = context.create(:issue, project: context.project)
{ issue: issue, merge_request: context.create_merge_request_closing_issue(issue) }
end,
start_time_conditions: [["merge request that closes issue is merged",
-> (context, data) do
context.merge_merge_requests_closing_issue(data[:issue])
end ]],
end_time_conditions: [["merge request that closes issue is deployed to production",
-> (context, data) do
context.deploy_master
end],
["production deploy happens after merge request is merged (along with other changes)",
lambda do |context, data|
# Make other changes on master
sha = context.project.repository.commit_file(
context.user,
context.random_git_name,
"content",
"commit message",
'master',
false)
context.project.repository.commit(sha)
context.deploy_master
end]])
context "when a regular merge request (that doesn't close the issue) is merged and deployed" do
it "returns nil" do
5.times do
merge_request = create(:merge_request)
MergeRequests::MergeService.new(project, user).execute(merge_request)
deploy_master
end
expect(subject.staging).to be_nil
end
end
context "when the deployment happens to a non-production environment" do
it "returns nil" do
5.times do
issue = create(:issue, project: project)
merge_request = create_merge_request_closing_issue(issue)
MergeRequests::MergeService.new(project, user).execute(merge_request)
deploy_master(environment: 'staging')
end
expect(subject.staging).to be_nil
end
end
end
require 'spec_helper'
describe CycleAnalytics::Summary, models: true do
let(:project) { create(:project) }
let(:from) { Time.now }
let(:user) { create(:user, :admin) }
subject { described_class.new(project, from: from) }
describe "#new_issues" do
it "finds the number of issues created after the 'from date'" do
Timecop.freeze(5.days.ago) { create(:issue, project: project) }
Timecop.freeze(5.days.from_now) { create(:issue, project: project) }
expect(subject.new_issues).to eq(1)
end
it "doesn't find issues from other projects" do
Timecop.freeze(5.days.from_now) { create(:issue, project: create(:project)) }
expect(subject.new_issues).to eq(0)
end
end
describe "#commits" do
it "finds the number of commits created after the 'from date'" do
Timecop.freeze(5.days.ago) { create_commit("Test message", project, user, 'master') }
Timecop.freeze(5.days.from_now) { create_commit("Test message", project, user, 'master') }
expect(subject.commits).to eq(1)
end
it "doesn't find commits from other projects" do
Timecop.freeze(5.days.from_now) { create_commit("Test message", create(:project), user, 'master') }
expect(subject.commits).to eq(0)
end
end
describe "#deploys" do
it "finds the number of deploys made created after the 'from date'" do
Timecop.freeze(5.days.ago) { create(:deployment, project: project) }
Timecop.freeze(5.days.from_now) { create(:deployment, project: project) }
expect(subject.deploys).to eq(1)
end
it "doesn't find commits from other projects" do
Timecop.freeze(5.days.from_now) { create(:deployment, project: create(:project)) }
expect(subject.deploys).to eq(0)
end
end
end
require 'spec_helper'
describe 'CycleAnalytics#test', feature: true do
extend CycleAnalyticsHelpers::TestGeneration
let(:project) { create(:project) }
let(:from_date) { 10.days.ago }
let(:user) { create(:user, :admin) }
subject { CycleAnalytics.new(project, from: from_date) }
generate_cycle_analytics_spec(
phase: :test,
data_fn: lambda do |context|
issue = context.create(:issue, project: context.project)
merge_request = context.create_merge_request_closing_issue(issue)
pipeline = context.create(:ci_pipeline, ref: merge_request.source_branch, sha: merge_request.diff_head_sha, project: context.project)
{ pipeline: pipeline, issue: issue }
end,
start_time_conditions: [["pipeline is started", -> (context, data) { data[:pipeline].run! }]],
end_time_conditions: [["pipeline is finished", -> (context, data) { data[:pipeline].succeed! }]],
post_fn: -> (context, data) do
context.merge_merge_requests_closing_issue(data[:issue])
context.deploy_master
end)
context "when the pipeline is for a regular merge request (that doesn't close an issue)" do
it "returns nil" do
5.times do
issue = create(:issue, project: project)
merge_request = create_merge_request_closing_issue(issue)
pipeline = create(:ci_pipeline, ref: "refs/heads/#{merge_request.source_branch}", sha: merge_request.diff_head_sha)
pipeline.run!
pipeline.succeed!
merge_merge_requests_closing_issue(issue)
deploy_master
end
expect(subject.test).to be_nil
end
end
context "when the pipeline is not for a merge request" do
it "returns nil" do
5.times do
pipeline = create(:ci_pipeline, ref: "refs/heads/master", sha: project.repository.commit('master').sha)
pipeline.run!
pipeline.succeed!
deploy_master
end
expect(subject.test).to be_nil
end
end
context "when the pipeline is dropped (failed)" do
it "returns nil" do
5.times do
issue = create(:issue, project: project)
merge_request = create_merge_request_closing_issue(issue)
pipeline = create(:ci_pipeline, ref: "refs/heads/#{merge_request.source_branch}", sha: merge_request.diff_head_sha)
pipeline.run!
pipeline.drop!
merge_merge_requests_closing_issue(issue)
deploy_master
end
expect(subject.test).to be_nil
end
end
context "when the pipeline is cancelled" do
it "returns nil" do
5.times do
issue = create(:issue, project: project)
merge_request = create_merge_request_closing_issue(issue)
pipeline = create(:ci_pipeline, ref: "refs/heads/#{merge_request.source_branch}", sha: merge_request.diff_head_sha)
pipeline.run!
pipeline.cancel!
merge_merge_requests_closing_issue(issue)
deploy_master
end
expect(subject.test).to be_nil
end
end
end
require 'spec_helper'
describe Issue::Metrics, models: true do
let(:project) { create(:project) }
subject { create(:issue, project: project) }
describe "when recording the default set of issue metrics on issue save" do
context "milestones" do
it "records the first time an issue is associated with a milestone" do
time = Time.now
Timecop.freeze(time) { subject.update(milestone: create(:milestone)) }
metrics = subject.metrics
expect(metrics).to be_present
expect(metrics.first_associated_with_milestone_at).to be_within(1.second).of(time)
end
it "does not record the second time an issue is associated with a milestone" do
time = Time.now
Timecop.freeze(time) { subject.update(milestone: create(:milestone)) }
Timecop.freeze(time + 2.hours) { subject.update(milestone: nil) }
Timecop.freeze(time + 6.hours) { subject.update(milestone: create(:milestone)) }
metrics = subject.metrics
expect(metrics).to be_present
expect(metrics.first_associated_with_milestone_at).to be_within(1.second).of(time)
end
end
context "list labels" do
it "records the first time an issue is associated with a list label" do
list_label = create(:label, lists: [create(:list)])
time = Time.now
Timecop.freeze(time) { subject.update(label_ids: [list_label.id]) }
metrics = subject.metrics
expect(metrics).to be_present
expect(metrics.first_added_to_board_at).to be_within(1.second).of(time)
end
it "does not record the second time an issue is associated with a list label" do
time = Time.now
first_list_label = create(:label, lists: [create(:list)])
Timecop.freeze(time) { subject.update(label_ids: [first_list_label.id]) }
second_list_label = create(:label, lists: [create(:list)])
Timecop.freeze(time + 5.hours) { subject.update(label_ids: [second_list_label.id]) }
metrics = subject.metrics
expect(metrics).to be_present
expect(metrics.first_added_to_board_at).to be_within(1.second).of(time)
end
end
end
end
require 'spec_helper'
describe MergeRequest::Metrics, models: true do
let(:project) { create(:project) }
subject { create(:merge_request, source_project: project) }
describe "when recording the default set of metrics on merge request save" do
it "records the merge time" do
time = Time.now
Timecop.freeze(time) { subject.mark_as_merged }
metrics = subject.metrics
expect(metrics).to be_present
expect(metrics.merged_at).to be_within(1.second).of(time)
end
end
end
...@@ -169,4 +169,83 @@ describe CreateDeploymentService, services: true do ...@@ -169,4 +169,83 @@ describe CreateDeploymentService, services: true do
end end
end end
end end
describe "merge request metrics" do
let(:params) do
{
environment: 'production',
ref: 'master',
tag: false,
sha: '97de212e80737a608d939f648d959671fb0a0142b',
}
end
let(:merge_request) { create(:merge_request, target_branch: 'master', source_branch: 'feature', source_project: project) }
context "while updating the 'first_deployed_to_production_at' time" do
before { merge_request.mark_as_merged }
context "for merge requests merged before the current deploy" do
it "sets the time if the deploy's environment is 'production'" do
time = Time.now
Timecop.freeze(time) { service.execute }
expect(merge_request.reload.metrics.first_deployed_to_production_at).to be_within(1.second).of(time)
end
it "doesn't set the time if the deploy's environment is not 'production'" do
staging_params = params.merge(environment: 'staging')
service = described_class.new(project, user, staging_params)
service.execute
expect(merge_request.reload.metrics.first_deployed_to_production_at).to be_nil
end
it 'does not raise errors if the merge request does not have a metrics record' do
merge_request.metrics.destroy
expect(merge_request.reload.metrics).to be_nil
expect { service.execute }.not_to raise_error
end
end
context "for merge requests merged before the previous deploy" do
context "if the 'first_deployed_to_production_at' time is already set" do
it "does not overwrite the older 'first_deployed_to_production_at' time" do
# Previous deploy
time = Time.now
Timecop.freeze(time) { service.execute }
expect(merge_request.reload.metrics.first_deployed_to_production_at).to be_within(1.second).of(time)
# Current deploy
service = described_class.new(project, user, params)
Timecop.freeze(time + 12.hours) { service.execute }
expect(merge_request.reload.metrics.first_deployed_to_production_at).to be_within(1.second).of(time)
end
end
context "if the 'first_deployed_to_production_at' time is not already set" do
it "does not overwrite the older 'first_deployed_to_production_at' time" do
# Previous deploy
time = 5.minutes.from_now
Timecop.freeze(time) { service.execute }
expect(merge_request.reload.metrics.merged_at).to be < merge_request.reload.metrics.first_deployed_to_production_at
merge_request.reload.metrics.update(first_deployed_to_production_at: nil)
expect(merge_request.reload.metrics.first_deployed_to_production_at).to be_nil
# Current deploy
service = described_class.new(project, user, params)
Timecop.freeze(time + 12.hours) { service.execute }
expect(merge_request.reload.metrics.first_deployed_to_production_at).to be_nil
end
end
end
end
end
end end
...@@ -339,6 +339,43 @@ describe GitPushService, services: true do ...@@ -339,6 +339,43 @@ describe GitPushService, services: true do
end end
end end
describe "issue metrics" do
let(:issue) { create :issue, project: project }
let(:commit_author) { create :user }
let(:commit) { project.commit }
let(:commit_time) { Time.now }
before do
project.team << [commit_author, :developer]
project.team << [user, :developer]
allow(commit).to receive_messages(
safe_message: "this commit \n mentions #{issue.to_reference}",
references: [issue],
author_name: commit_author.name,
author_email: commit_author.email,
committed_date: commit_time
)
allow(project.repository).to receive(:commits_between).and_return([commit])
end
context "while saving the 'first_mentioned_in_commit_at' metric for an issue" do
it 'sets the metric for referenced issues' do
execute_service(project, user, @oldrev, @newrev, @ref)
expect(issue.reload.metrics.first_mentioned_in_commit_at).to be_within(1.second).of(commit_time)
end
it 'does not set the metric for non-referenced issues' do
non_referenced_issue = create(:issue, project: project)
execute_service(project, user, @oldrev, @newrev, @ref)
expect(non_referenced_issue.reload.metrics.first_mentioned_in_commit_at).to be_nil
end
end
end
describe "closing issues from pushed commits containing a closing reference" do describe "closing issues from pushed commits containing a closing reference" do
let(:issue) { create :issue, project: project } let(:issue) { create :issue, project: project }
let(:other_issue) { create :issue, project: project } let(:other_issue) { create :issue, project: project }
......
...@@ -83,5 +83,34 @@ describe MergeRequests::CreateService, services: true do ...@@ -83,5 +83,34 @@ describe MergeRequests::CreateService, services: true do
} }
end end
end end
context 'while saving references to issues that the created merge request closes' do
let(:first_issue) { create(:issue, project: project) }
let(:second_issue) { create(:issue, project: project) }
let(:opts) do
{
title: 'Awesome merge_request',
source_branch: 'feature',
target_branch: 'master',
force_remove_source_branch: '1'
}
end
before do
project.team << [user, :master]
project.team << [assignee, :developer]
end
it 'creates a `MergeRequestsClosingIssues` record for each issue' do
issue_closing_opts = opts.merge(description: "Closes #{first_issue.to_reference} and #{second_issue.to_reference}")
service = described_class.new(project, user, issue_closing_opts)
allow(service).to receive(:execute_hooks)
merge_request = service.execute
issue_ids = MergeRequestsClosingIssues.where(merge_request: merge_request).pluck(:issue_id)
expect(issue_ids).to match_array([first_issue.id, second_issue.id])
end
end
end end
end end
...@@ -174,6 +174,58 @@ describe MergeRequests::RefreshService, services: true do ...@@ -174,6 +174,58 @@ describe MergeRequests::RefreshService, services: true do
end end
end end
context 'merge request metrics' do
let(:issue) { create :issue, project: @project }
let(:commit_author) { create :user }
let(:commit) { project.commit }
before do
project.team << [commit_author, :developer]
project.team << [user, :developer]
allow(commit).to receive_messages(
safe_message: "Closes #{issue.to_reference}",
references: [issue],
author_name: commit_author.name,
author_email: commit_author.email,
committed_date: Time.now
)
allow_any_instance_of(MergeRequest).to receive(:commits).and_return([commit])
end
context 'when the merge request is sourced from the same project' do
it 'creates a `MergeRequestsClosingIssues` record for each issue closed by a commit' do
merge_request = create(:merge_request, target_branch: 'master', source_branch: 'feature', source_project: @project)
refresh_service = service.new(@project, @user)
allow(refresh_service).to receive(:execute_hooks)
refresh_service.execute(@oldrev, @newrev, 'refs/heads/feature')
issue_ids = MergeRequestsClosingIssues.where(merge_request: merge_request).pluck(:issue_id)
expect(issue_ids).to eq([issue.id])
end
end
context 'when the merge request is sourced from a different project' do
it 'creates a `MergeRequestsClosingIssues` record for each issue closed by a commit' do
forked_project = create(:project)
create(:forked_project_link, forked_to_project: forked_project, forked_from_project: @project)
merge_request = create(:merge_request,
target_branch: 'master',
source_branch: 'feature',
target_project: @project,
source_project: forked_project)
refresh_service = service.new(@project, @user)
allow(refresh_service).to receive(:execute_hooks)
refresh_service.execute(@oldrev, @newrev, 'refs/heads/feature')
issue_ids = MergeRequestsClosingIssues.where(merge_request: merge_request).pluck(:issue_id)
expect(issue_ids).to eq([issue.id])
end
end
end
def reload_mrs def reload_mrs
@merge_request.reload @merge_request.reload
@fork_merge_request.reload @fork_merge_request.reload
......
...@@ -263,5 +263,42 @@ describe MergeRequests::UpdateService, services: true do ...@@ -263,5 +263,42 @@ describe MergeRequests::UpdateService, services: true do
end end
end end
end end
context 'while saving references to issues that the updated merge request closes' do
let(:first_issue) { create(:issue, project: project) }
let(:second_issue) { create(:issue, project: project) }
it 'creates a `MergeRequestsClosingIssues` record for each issue' do
issue_closing_opts = { description: "Closes #{first_issue.to_reference} and #{second_issue.to_reference}" }
service = described_class.new(project, user, issue_closing_opts)
allow(service).to receive(:execute_hooks)
service.execute(merge_request)
issue_ids = MergeRequestsClosingIssues.where(merge_request: merge_request).pluck(:issue_id)
expect(issue_ids).to match_array([first_issue.id, second_issue.id])
end
it 'removes `MergeRequestsClosingIssues` records when issues are not closed anymore' do
opts = {
title: 'Awesome merge_request',
description: "Closes #{first_issue.to_reference} and #{second_issue.to_reference}",
source_branch: 'feature',
target_branch: 'master',
force_remove_source_branch: '1'
}
merge_request = MergeRequests::CreateService.new(project, user, opts).execute
issue_ids = MergeRequestsClosingIssues.where(merge_request: merge_request).pluck(:issue_id)
expect(issue_ids).to match_array([first_issue.id, second_issue.id])
service = described_class.new(project, user, description: "not closing any issues")
allow(service).to receive(:execute_hooks)
service.execute(merge_request.reload)
issue_ids = MergeRequestsClosingIssues.where(merge_request: merge_request).pluck(:issue_id)
expect(issue_ids).to be_empty
end
end
end end
end end
...@@ -379,6 +379,7 @@ describe NotificationService, services: true do ...@@ -379,6 +379,7 @@ describe NotificationService, services: true do
it "emails subscribers of the issue's labels" do it "emails subscribers of the issue's labels" do
subscriber = create(:user) subscriber = create(:user)
label = create(:label, issues: [issue]) label = create(:label, issues: [issue])
issue.reload
label.toggle_subscription(subscriber) label.toggle_subscription(subscriber)
notification.new_issue(issue, @u_disabled) notification.new_issue(issue, @u_disabled)
...@@ -399,6 +400,7 @@ describe NotificationService, services: true do ...@@ -399,6 +400,7 @@ describe NotificationService, services: true do
project.team << [guest, :guest] project.team << [guest, :guest]
label = create(:label, issues: [confidential_issue]) label = create(:label, issues: [confidential_issue])
confidential_issue.reload
label.toggle_subscription(non_member) label.toggle_subscription(non_member)
label.toggle_subscription(author) label.toggle_subscription(author)
label.toggle_subscription(assignee) label.toggle_subscription(assignee)
......
module CycleAnalyticsHelpers
def create_commit_referencing_issue(issue, branch_name: random_git_name)
project.repository.add_branch(user, branch_name, 'master')
create_commit("Commit for ##{issue.iid}", issue.project, user, branch_name)
end
def create_commit(message, project, user, branch_name)
filename = random_git_name
oldrev = project.repository.commit(branch_name).sha
options = {
committer: project.repository.user_to_committer(user),
author: project.repository.user_to_committer(user),
commit: { message: message, branch: branch_name, update_ref: true },
file: { content: "content", path: filename, update: false }
}
commit_sha = Gitlab::Git::Blob.commit(project.repository, options)
project.repository.commit(commit_sha)
GitPushService.new(project,
user,
oldrev: oldrev,
newrev: commit_sha,
ref: 'refs/heads/master').execute
end
def create_merge_request_closing_issue(issue, message: nil, source_branch: nil)
if !source_branch || project.repository.commit(source_branch).blank?
source_branch = random_git_name
project.repository.add_branch(user, source_branch, 'master')
end
sha = project.repository.commit_file(user, random_git_name, "content", "commit message", source_branch, false)
project.repository.commit(sha)
opts = {
title: 'Awesome merge_request',
description: message || "Fixes #{issue.to_reference}",
source_branch: source_branch,
target_branch: 'master'
}
MergeRequests::CreateService.new(project, user, opts).execute
end
def merge_merge_requests_closing_issue(issue)
merge_requests = issue.closed_by_merge_requests
merge_requests.each { |merge_request| MergeRequests::MergeService.new(project, user).execute(merge_request) }
end
def deploy_master(environment: 'production')
CreateDeploymentService.new(project, user, {
environment: environment,
ref: 'master',
tag: false,
sha: project.repository.commit('master').sha
}).execute
end
end
RSpec.configure do |config|
config.include CycleAnalyticsHelpers
end
# rubocop:disable Metrics/AbcSize
# Note: The ABC size is large here because we have a method generating test cases with
# multiple nested contexts. This shouldn't count as a violation.
module CycleAnalyticsHelpers
module TestGeneration
# Generate the most common set of specs that all cycle analytics phases need to have.
#
# Arguments:
#
# phase: Which phase are we testing? Will call `CycleAnalytics.new.send(phase)` for the final assertion
# data_fn: A function that returns a hash, constituting initial data for the test case
# start_time_conditions: An array of `conditions`. Each condition is an tuple of `condition_name` and `condition_fn`. `condition_fn` is called with
# `context` (no lexical scope, so need to do `context.create` for factories, for example) and `data` (from the `data_fn`).
# Each `condition_fn` is expected to implement a case which consitutes the start of the given cycle analytics phase.
# end_time_conditions: An array of `conditions`. Each condition is an tuple of `condition_name` and `condition_fn`. `condition_fn` is called with
# `context` (no lexical scope, so need to do `context.create` for factories, for example) and `data` (from the `data_fn`).
# Each `condition_fn` is expected to implement a case which consitutes the end of the given cycle analytics phase.
# before_end_fn: This function is run before calling the end time conditions. Used for setup that needs to be run between the start and end conditions.
# post_fn: Code that needs to be run after running the end time conditions.
def generate_cycle_analytics_spec(phase:, data_fn:, start_time_conditions:, end_time_conditions:, before_end_fn: nil, post_fn: nil)
combinations_of_start_time_conditions = (1..start_time_conditions.size).flat_map { |size| start_time_conditions.combination(size).to_a }
combinations_of_end_time_conditions = (1..end_time_conditions.size).flat_map { |size| end_time_conditions.combination(size).to_a }
scenarios = combinations_of_start_time_conditions.product(combinations_of_end_time_conditions)
scenarios.each do |start_time_conditions, end_time_conditions|
context "start condition: #{start_time_conditions.map(&:first).to_sentence}" do
context "end condition: #{end_time_conditions.map(&:first).to_sentence}" do
it "finds the median of available durations between the two conditions" do
time_differences = Array.new(5) do |index|
data = data_fn[self]
start_time = (index * 10).days.from_now
end_time = start_time + rand(1..5).days
start_time_conditions.each do |condition_name, condition_fn|
Timecop.freeze(start_time) { condition_fn[self, data] }
end
# Run `before_end_fn` at the midpoint between `start_time` and `end_time`
Timecop.freeze(start_time + (end_time - start_time) / 2) { before_end_fn[self, data] } if before_end_fn
end_time_conditions.each do |condition_name, condition_fn|
Timecop.freeze(end_time) { condition_fn[self, data] }
end
Timecop.freeze(end_time + 1.day) { post_fn[self, data] } if post_fn
end_time - start_time
end
median_time_difference = time_differences.sort[2]
expect(subject.send(phase)).to be_within(5).of(median_time_difference)
end
context "when the data belongs to another project" do
let(:other_project) { create(:project) }
it "returns nil" do
# Use a stub to "trick" the data/condition functions
# into using another project. This saves us from having to
# define separate data/condition functions for this particular
# test case.
allow(self).to receive(:project) { other_project }
5.times do
data = data_fn[self]
start_time = Time.now
end_time = rand(1..10).days.from_now
start_time_conditions.each do |condition_name, condition_fn|
Timecop.freeze(start_time) { condition_fn[self, data] }
end
end_time_conditions.each do |condition_name, condition_fn|
Timecop.freeze(end_time) { condition_fn[self, data] }
end
Timecop.freeze(end_time + 1.day) { post_fn[self, data] } if post_fn
end
# Turn off the stub before checking assertions
allow(self).to receive(:project).and_call_original
expect(subject.send(phase)).to be_nil
end
end
context "when the end condition happens before the start condition" do
it 'returns nil' do
data = data_fn[self]
start_time = Time.now
end_time = start_time + rand(1..5).days
# Run `before_end_fn` at the midpoint between `start_time` and `end_time`
Timecop.freeze(start_time + (end_time - start_time) / 2) { before_end_fn[self, data] } if before_end_fn
end_time_conditions.each do |condition_name, condition_fn|
Timecop.freeze(start_time) { condition_fn[self, data] }
end
start_time_conditions.each do |condition_name, condition_fn|
Timecop.freeze(end_time) { condition_fn[self, data] }
end
Timecop.freeze(end_time + 1.day) { post_fn[self, data] } if post_fn
expect(subject.send(phase)).to be_nil
end
end
end
end
context "start condition NOT PRESENT: #{start_time_conditions.map(&:first).to_sentence}" do
context "end condition: #{end_time_conditions.map(&:first).to_sentence}" do
it "returns nil" do
5.times do
data = data_fn[self]
end_time = rand(1..10).days.from_now
end_time_conditions.each_with_index do |(condition_name, condition_fn), index|
Timecop.freeze(end_time + index.days) { condition_fn[self, data] }
end
Timecop.freeze(end_time + 1.day) { post_fn[self, data] } if post_fn
end
expect(subject.send(phase)).to be_nil
end
end
end
context "start condition: #{start_time_conditions.map(&:first).to_sentence}" do
context "end condition NOT PRESENT: #{end_time_conditions.map(&:first).to_sentence}" do
it "returns nil" do
5.times do
data = data_fn[self]
start_time = Time.now
start_time_conditions.each do |condition_name, condition_fn|
Timecop.freeze(start_time) { condition_fn[self, data] }
end
post_fn[self, data] if post_fn
end
expect(subject.send(phase)).to be_nil
end
end
end
end
context "when none of the start / end conditions are matched" do
it "returns nil" do
expect(subject.send(phase)).to be_nil
end
end
end
end
end
module GitHelpers
def random_git_name
"#{FFaker::Product.brand}-#{FFaker::Product.brand}-#{rand(1000)}"
end
end
RSpec.configure do |config|
config.include GitHelpers
end
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment