Commit 969286fc authored by henrik's avatar henrik

Merge branch 'master' of https://gitlab.com/gitlab-org/gitlab-ce

parents feb3913f 3c31f185
......@@ -19,8 +19,6 @@ variables:
before_script:
- source ./scripts/prepare_build.sh
- cp config/gitlab.yml.example config/gitlab.yml
- mkdir -p tmp/tests
- mount -t tmpfs tmpfs tmp/tests || echo "tmpfs mount failed, falling back to disc"
- bundle --version
- '[ "$USE_BUNDLE_INSTALL" != "true" ] || retry bundle install --without postgres production --jobs $(nproc) "${FLAGS[@]}"'
- retry gem install knapsack
......
Please view this file on the master branch, on stable branches it's out of date.
v 8.13.0 (unreleased)
- Respond with 404 Not Found for non-existent tags (Linus Thiel)
- Truncate long labels with ellipsis in labels page
- Bump mail_room to v0.8.1 to fix thread cleanup issue
- Update runner version only when updating contacted_at
- Add link from system note to compare with previous version
- Improve issue load time performance by avoiding ORDER BY in find_by call
- Use gitlab-shell v3.6.2 (GIT TRACE logging)
- Add `/projects/visible` API endpoint (Ben Boeckel)
- Fix centering of custom header logos (Ashley Dumaine)
- ExpireBuildArtifactsWorker query builds table without ordering enqueuing one job per build to cleanup
- Add an example for testing a phoenix application with Gitlab CI in the docs (Manthan Mallikarjun)
- Updating verbiage on git basics to be more intuitive
- Clarify documentation for Runners API (Gennady Trafimenkov)
- Change user & group landing page routing from /u/:username to /:username
- Prevent running GfmAutocomplete setup for each diff note !6569
- Added documentation for .gitattributes files
- AbstractReferenceFilter caches project_refs on RequestStore when active
- Replaced the check sign to arrow in the show build view. !6501
- Add a /wip slash command to toggle the Work In Progress status of a merge request. !6259 (tbalthazar)
- Fix Error 500 when viewing old merge requests with bad diff data
- Create a new /templates namespace for the /licenses, /gitignores and /gitlab_ci_ymls API endpoints. !5717 (tbalthazar)
- Speed-up group milestones show page
- Fix inconsistent options dropdown caret on mobile viewports (ClemMakesApps)
- Don't include archived projects when creating group milestones. !4940 (Jeroen Jacobs)
- Add tag shortcut from the Commit page. !6543
- Keep refs for each deployment
- Allow browsing branches that end with '.atom'
- Log LDAP lookup errors and don't swallow unrelated exceptions. !6103 (Markus Koller)
- Add more tests for calendar contribution (ClemMakesApps)
- Update Gitlab Shell to fix some problems with moving projects between storages
......@@ -42,6 +48,7 @@ v 8.13.0 (unreleased)
- Fix todos page mobile viewport layout (ClemMakesApps)
- Fix inconsistent highlighting of already selected activity nav-links (ClemMakesApps)
- Remove redundant mixins (ClemMakesApps)
- Added 'Download' button to the Snippets page (Justin DiPierro)
- Fix robots.txt disallowing access to groups starting with "s" (Matt Harrison)
- Close open merge request without source project (Katarzyna Kobierska Ula Budziszewska)
- Fix that manual jobs would no longer block jobs in the next stage. !6604
......@@ -65,8 +72,10 @@ v 8.13.0 (unreleased)
- Fix Long commit messages overflow viewport in file tree
- Revert avoid touching file system on Build#artifacts?
- Stop using a Redis lease when updating the project activity timestamp whenever a new event is created
- Add disabled delete button to protected branches (ClemMakesApps)
- Add broadcast messages and alerts below sub-nav
- Better empty state for Groups view
- API: New /users/:id/events endpoint
- Update ruby-prof to 0.16.2. !6026 (Elan Ruusamäe)
- Replace bootstrap caret with fontawesome caret (ClemMakesApps)
- Fix unnecessary escaping of reserved HTML characters in milestone title. !6533
......@@ -79,23 +88,33 @@ v 8.13.0 (unreleased)
- API: expose pipeline data in builds API (!6502, Guilherme Salazar)
- Notify the Merger about merge after successful build (Dimitris Karakasilis)
- Reorder issue and merge request titles to show IDs first. !6503 (Greg Laubenstein)
- Add a new gitlab:users:clear_all_authentication_tokens task. !6745
- Reduce queries needed to find users using their SSH keys when pushing commits
- Prevent rendering the link to all when the author has no access (Katarzyna Kobierska Ula Budziszewska)
- Fix broken repository 500 errors in project list
- Fix Pipeline list commit column width should be adjusted
- Close todos when accepting merge requests via the API !6486 (tonygambone)
- Ability to batch assign issues relating to a merge request to the author. !5725 (jamedjo)
- Changed Slack service user referencing from full name to username (Sebastian Poxhofer)
- Retouch environments list and deployments list
- Add multiple command support for all label related slash commands !6780 (barthc)
- Add Container Registry on/off status to Admin Area !6638 (the-undefined)
- Allow empty merge requests !6384 (Artem Sidorenko)
- Grouped pipeline dropdown is a scrollable container
- Cleanup Ci::ApplicationController. !6757 (Takuya Noguchi)
- Fix a typo in doc/api/labels.md
- API: all unknown routing will be handled with 404 Not Found
- Make guests unable to view MRs on private projects
v 8.12.6
- Update mailroom to 0.8.1 in Gemfile.lock !6814
v 8.12.5 (unreleased)
- Update the mail_room gem to 0.8.1 to fix a race condition with the mailbox watching thread
v 8.12.5
- Switch from request to env in ::API::Helpers. !6615
- Update the mail_room gem to 0.8.1 to fix a race condition with the mailbox watching thread. !6714
- Improve issue load time performance by avoiding ORDER BY in find_by call. !6724
- Add a new gitlab:users:clear_all_authentication_tokens task. !6745
- Don't send Private-Token (API authentication) headers to Sentry
- Share projects via the API only with groups the authenticated user can access
v 8.12.4
- Fix "Copy to clipboard" tooltip to say "Copied!" when clipboard button is clicked. !6294 (lukehowell)
......@@ -110,7 +129,6 @@ v 8.12.4
- Fix failed project deletion when feature visibility set to private. !6688
- Prevent claiming associated model IDs via import.
- Set GitLab project exported file permissions to owner only
- Change user & group landing page routing from /u/:username to /:username
v 8.12.3
- Update Gitlab Shell to support low IO priority for storage moves
......@@ -130,13 +148,13 @@ v 8.12.2
- Fix bug where 'Search results' repeated many times when a search in the emoji search form is cleared (Xavier Bick) (@zeiv)
- Fix resolve discussion buttons endpoint path
- Refactor remnants of CoffeeScript destructured opts and super !6261
- Prevent running GfmAutocomplete setup for each diff note !6569
v 8.12.1
- Fix a memory leak in HTML::Pipeline::SanitizationFilter::WHITELIST
- Fix issue with search filter labels not displaying
v 8.12.0
- Removes inconsistency regarding tagging immediatelly as merged once you create a new branch. !6408
- Update the rouge gem to 2.0.6, which adds highlighting support for JSX, Prometheus, and others. !6251
- Only check :can_resolve permission if the note is resolvable
- Bump fog-aws to v0.11.0 to support ap-south-1 region
......@@ -324,6 +342,10 @@ v 8.12.0
- Fix non-master branch readme display in tree view
- Add UX improvements for merge request version diffs
v 8.11.9
- Don't send Private-Token (API authentication) headers to Sentry
- Share projects via the API only with groups the authenticated user can access
v 8.11.8
- Respect the fork_project permission when forking projects
- Set a restrictive CORS policy on the API for credentialed requests
......@@ -334,7 +356,6 @@ v 8.11.7
- Avoid conflict with admin labels when importing GitHub labels. !6158
- Restores `fieldName` to allow only string values in `gl_dropdown.js`. !6234
- Allow the Rails cookie to be used for API authentication.
- Updating verbiage on git basics to be more intuitive
v 8.11.6
- Fix unnecessary horizontal scroll area in pipeline visualizations. !6005
......@@ -495,7 +516,6 @@ v 8.11.0
- Add pipeline events hook
- Bump gitlab_git to speedup DiffCollection iterations
- Rewrite description of a blocked user in admin settings. (Elias Werberich)
- Clarify documentation for Runners API (Gennady Trafimenkov)
- Make branches sortable without push permission !5462 (winniehell)
- Check for Ci::Build artifacts at database level on pipeline partial
- Convert image diff background image to CSS (ClemMakesApps)
......@@ -551,6 +571,10 @@ v 8.11.0
- Update gitlab_git gem to 10.4.7
- Simplify SQL queries of marking a todo as done
v 8.10.12
- Don't send Private-Token (API authentication) headers to Sentry
- Share projects via the API only with groups the authenticated user can access
v 8.10.11
- Respect the fork_project permission when forking projects
- Set a restrictive CORS policy on the API for credentialed requests
......
......@@ -262,6 +262,8 @@ group :development do
# thin instead webrick
gem 'thin', '~> 1.7.0'
gem 'activerecord_sane_schema_dumper', '0.2'
end
group :development, :test do
......
......@@ -38,6 +38,8 @@ GEM
multi_json (~> 1.11, >= 1.11.2)
rack (>= 1.5.2, < 3)
railties (>= 4.0, < 5.1)
activerecord_sane_schema_dumper (0.2)
rails (>= 4, < 5)
activesupport (4.2.7.1)
i18n (~> 0.7)
json (~> 1.7, >= 1.7.7)
......@@ -805,6 +807,7 @@ DEPENDENCIES
RedCloth (~> 4.3.2)
ace-rails-ap (~> 4.1.0)
activerecord-session_store (~> 1.0.0)
activerecord_sane_schema_dumper (= 0.2)
acts-as-taggable-on (~> 4.0)
addressable (~> 2.3.8)
after_commit_queue (~> 1.3.0)
......
......@@ -6,11 +6,10 @@
groupProjectsPath: "/api/:version/groups/:id/projects.json",
projectsPath: "/api/:version/projects.json?simple=true",
labelsPath: "/:namespace_path/:project_path/labels",
licensePath: "/api/:version/licenses/:key",
gitignorePath: "/api/:version/gitignores/:key",
gitlabCiYmlPath: "/api/:version/gitlab_ci_ymls/:key",
licensePath: "/api/:version/templates/licenses/:key",
gitignorePath: "/api/:version/templates/gitignores/:key",
gitlabCiYmlPath: "/api/:version/templates/gitlab_ci_ymls/:key",
issuableTemplatePath: "/:namespace_path/:project_path/templates/:type/:key",
group: function(group_id, callback) {
var url = Api.buildUrl(Api.groupPath)
.replace(':id', group_id);
......
......@@ -10,7 +10,7 @@ class Projects::MergeRequestsController < Projects::ApplicationController
before_action :module_enabled
before_action :merge_request, only: [
:edit, :update, :show, :diffs, :commits, :conflicts, :builds, :pipelines, :merge, :merge_check,
:ci_status, :toggle_subscription, :cancel_merge_when_build_succeeds, :remove_wip, :resolve_conflicts
:ci_status, :toggle_subscription, :cancel_merge_when_build_succeeds, :remove_wip, :resolve_conflicts, :assign_related_issues
]
before_action :validates_merge_request, only: [:show, :diffs, :commits, :builds, :pipelines]
before_action :define_show_vars, only: [:show, :diffs, :commits, :conflicts, :builds, :pipelines]
......@@ -31,6 +31,8 @@ class Projects::MergeRequestsController < Projects::ApplicationController
# Allow modify merge_request
before_action :authorize_update_merge_request!, only: [:close, :edit, :update, :remove_wip, :sort]
before_action :authenticate_user!, only: [:assign_related_issues]
before_action :authorize_can_resolve_conflicts!, only: [:conflicts, :resolve_conflicts]
def index
......@@ -354,6 +356,25 @@ class Projects::MergeRequestsController < Projects::ApplicationController
render layout: false
end
def assign_related_issues
result = MergeRequests::AssignIssuesService.new(project, current_user, merge_request: @merge_request).execute
respond_to do |format|
format.html do
case result[:count]
when 0
flash[:error] = "Failed to assign you issues related to the merge request"
when 1
flash[:notice] = "1 issue has been assigned to you"
else
flash[:notice] = "#{result[:count]} issues have been assigned to you"
end
redirect_to(merge_request_path(@merge_request))
end
end
end
def ci_status
pipeline = @merge_request.pipeline
if pipeline
......
......@@ -20,6 +20,8 @@ class Projects::TagsController < Projects::ApplicationController
def show
@tag = @repository.find_tag(params[:id])
return render_404 unless @tag
@release = @project.releases.find_or_initialize_by(tag: @tag.name)
@commit = @repository.commit(@tag.target)
end
......
class SnippetsController < ApplicationController
include ToggleAwardEmoji
before_action :snippet, only: [:show, :edit, :destroy, :update, :raw]
before_action :snippet, only: [:show, :edit, :destroy, :update, :raw, :download]
# Allow read snippet
before_action :authorize_read_snippet!, only: [:show, :raw]
before_action :authorize_read_snippet!, only: [:show, :raw, :download]
# Allow modify snippet
before_action :authorize_update_snippet!, only: [:edit, :update]
......@@ -12,7 +12,7 @@ class SnippetsController < ApplicationController
# Allow destroy snippet
before_action :authorize_admin_snippet!, only: [:destroy]
skip_before_action :authenticate_user!, only: [:index, :show, :raw]
skip_before_action :authenticate_user!, only: [:index, :show, :raw, :download]
layout 'snippets'
respond_to :html
......@@ -75,6 +75,14 @@ class SnippetsController < ApplicationController
)
end
def download
send_data(
@snippet.content,
type: 'text/plain; charset=utf-8',
filename: @snippet.sanitized_file_name
)
end
protected
def snippet
......
......@@ -72,6 +72,19 @@ module MergeRequestsHelper
)
end
def mr_assign_issues_link
issues = MergeRequests::AssignIssuesService.new(@project,
current_user,
merge_request: @merge_request,
closes_issues: mr_closes_issues
).assignable_issues
path = assign_related_issues_namespace_project_merge_request_path(@project.namespace, @project, @merge_request)
if issues.present?
pluralize_this_issue = issues.count > 1 ? "these issues" : "this issue"
link_to "Assign yourself to #{pluralize_this_issue}", path, method: :post
end
end
def source_branch_with_namespace(merge_request)
branch = link_to(merge_request.source_branch, namespace_project_commits_path(merge_request.source_project.namespace, merge_request.source_project, merge_request.source_branch))
......
......@@ -68,8 +68,10 @@ class Event < ActiveRecord::Base
true
elsif issue? || issue_note?
Ability.allowed?(user, :read_issue, note? ? note_target : target)
elsif merge_request? || merge_request_note?
Ability.allowed?(user, :read_merge_request, note? ? note_target : target)
else
((merge_request? || note?) && target.present?) || milestone?
milestone?
end
end
......@@ -280,6 +282,10 @@ class Event < ActiveRecord::Base
note? && target && target.for_issue?
end
def merge_request_note?
note? && target && target.for_merge_request?
end
def project_snippet_note?
target.for_snippet?
end
......
......@@ -10,7 +10,7 @@ class ProjectGroupLink < ActiveRecord::Base
belongs_to :group
validates :project_id, presence: true
validates :group_id, presence: true
validates :group, presence: true
validates :group_id, uniqueness: { scope: [:project_id], message: "already shared with this group" }
validates :group_access, presence: true
validates :group_access, inclusion: { in: Gitlab::Access.values }, presence: true
......
......@@ -1016,7 +1016,8 @@ class Repository
root_ref_commit = commit(root_ref)
if branch_commit
is_ancestor?(branch_commit.id, root_ref_commit.id)
same_head = branch_commit.id == root_ref_commit.id
!same_head && is_ancestor?(branch_commit.id, root_ref_commit.id)
else
nil
end
......
......@@ -40,7 +40,6 @@ class ProjectPolicy < BasePolicy
can! :read_milestone
can! :read_project_snippet
can! :read_project_member
can! :read_merge_request
can! :read_note
can! :create_project
can! :create_issue
......@@ -63,6 +62,7 @@ class ProjectPolicy < BasePolicy
can! :read_pipeline
can! :read_environment
can! :read_deployment
can! :read_merge_request
end
# Permissions given when an user is team member of a project
......@@ -117,6 +117,7 @@ class ProjectPolicy < BasePolicy
can! :read_container_image
can! :build_download_code
can! :build_read_container_image
can! :read_merge_request
end
def owner_access!
......
module MergeRequests
class AssignIssuesService < BaseService
def assignable_issues
@assignable_issues ||= begin
if current_user == merge_request.author
closes_issues.select do |issue|
!issue.assignee_id? && can?(current_user, :admin_issue, issue)
end
else
[]
end
end
end
def execute
assignable_issues.each do |issue|
Issues::UpdateService.new(issue.project, current_user, assignee_id: current_user.id).execute(issue)
end
{
count: assignable_issues.count
}
end
private
def merge_request
params[:merge_request]
end
def closes_issues
@closes_issues ||= params[:closes_issues] || merge_request.closes_issues(current_user)
end
end
end
......@@ -475,10 +475,12 @@ class NotificationService
end
def reject_users_without_access(recipients, target)
return recipients unless target.is_a?(Issue)
return recipients unless target.is_a?(Issuable)
ability = :"read_#{target.to_ability_name}"
recipients.select do |user|
user.can?(:read_issue, target)
user.can?(ability, target)
end
end
......
......@@ -122,7 +122,12 @@ module SlashCommands
command :label do |labels_param|
label_ids = find_label_ids(labels_param)
@updates[:add_label_ids] = label_ids unless label_ids.empty?
if label_ids.any?
@updates[:add_label_ids] ||= []
@updates[:add_label_ids] += label_ids
@updates[:add_label_ids].uniq!
end
end
desc 'Remove all or specific label(s)'
......@@ -136,7 +141,12 @@ module SlashCommands
if labels_param.present?
label_ids = find_label_ids(labels_param)
@updates[:remove_label_ids] = label_ids unless label_ids.empty?
if label_ids.any?
@updates[:remove_label_ids] ||= []
@updates[:remove_label_ids] += label_ids
@updates[:remove_label_ids].uniq!
end
else
@updates[:label_ids] = []
end
......@@ -152,7 +162,12 @@ module SlashCommands
command :relabel do |labels_param|
label_ids = find_label_ids(labels_param)
@updates[:label_ids] = label_ids unless label_ids.empty?
if label_ids.any?
@updates[:label_ids] ||= []
@updates[:label_ids] += label_ids
@updates[:label_ids].uniq!
end
end
desc 'Add a todo'
......
......@@ -273,12 +273,12 @@ class TodoService
end
def reject_users_without_access(users, project, target)
if target.is_a?(Note) && target.for_issue?
if target.is_a?(Note) && (target.for_issue? || target.for_merge_request?)
target = target.noteable
end
if target.is_a?(Issue)
select_users(users, :read_issue, target)
if target.is_a?(Issuable)
select_users(users, :"read_#{target.to_ability_name}", target)
else
select_users(users, :read_project, project)
end
......
......@@ -221,7 +221,11 @@
%fieldset
%legend Metrics
%p
These settings require a restart to take effect.
Setup InfluxDB to measure a wide variety of statistics like the time spent
in running SQL queries. These settings require a
= link_to 'restart', help_page_path('administration/restart_gitlab')
to take effect.
= link_to icon('question-circle'), help_page_path('administration/monitoring/performance/introduction')
.form-group
.col-sm-offset-2.col-sm-10
.checkbox
......
......@@ -30,8 +30,8 @@
= render 'projects/buttons/download', project: @project, ref: branch.name
- if can_remove_branch?(@project, branch.name)
= link_to namespace_project_branch_path(@project.namespace, @project, branch.name), class: 'btn btn-remove remove-row has-tooltip', title: "Delete branch", method: :delete, data: { confirm: "Deleting the '#{branch.name}' branch cannot be undone. Are you sure?", container: 'body' }, remote: true do
- if can?(current_user, :push_code, @project)
= link_to namespace_project_branch_path(@project.namespace, @project, branch.name), class: "btn btn-remove remove-row has-tooltip #{can_remove_branch?(@project, branch.name) ? '' : 'disabled'}", title: "Delete branch", method: :delete, data: { confirm: "Deleting the '#{branch.name}' branch cannot be undone. Are you sure?", container: 'body' }, remote: true do
= icon("trash-o")
- if branch.name != @repository.root_ref
......
......@@ -91,7 +91,7 @@
Git Large File Storage
= link_to icon('question-circle'), help_page_path('workflow/lfs/manage_large_binaries_with_git_lfs')
.col-md-3
= f.select :lfs_enabled, [%w(Enabled true), %w(Disabled false)], {}, selected: @project.lfs_enabled?, class: 'pull-right form-control'
= f.select :lfs_enabled, [%w(Enabled true), %w(Disabled false)], {}, selected: @project.lfs_enabled?, class: 'pull-right form-control', data: { field: 'lfs_enabled' }
- if Gitlab.config.registry.enabled
.form-group
......
......@@ -35,3 +35,4 @@
Accepting this merge request will close #{"issue".pluralize(mr_closes_issues.size)}
= succeed '.' do
!= markdown issues_sentence(mr_closes_issues), pipeline: :gfm, author: @merge_request.author
= mr_assign_issues_link
......@@ -9,6 +9,7 @@
.file-actions
= clipboard_button(clipboard_target: ".blob-content[data-blob-id='#{@snippet.id}']")
= link_to 'Raw', raw_snippet_path(@snippet), class: "btn btn-sm", target: "_blank"
= link_to 'Download', download_snippet_path(@snippet), class: "btn btn-sm"
= render 'shared/snippets/blob'
= render 'award_emoji/awards_block', awardable: @snippet, inline: true
\ No newline at end of file
......@@ -159,7 +159,7 @@ resources :namespaces, path: '/', constraints: { id: /[a-zA-Z.0-9_\-]+/ }, only:
get(
'/commits/*id',
to: 'commits#show',
constraints: { id: /(?:[^.]|\.(?!atom$))+/, format: /atom/ },
constraints: { id: /.+/, format: false },
as: :commits
)
end
......@@ -277,6 +277,7 @@ resources :namespaces, path: '/', constraints: { id: /[a-zA-Z.0-9_\-]+/ }, only:
post :remove_wip
get :diff_for_path
post :resolve_conflicts
post :assign_related_issues
end
collection do
......
resources :snippets, concerns: :awardable do
member do
get 'raw'
get 'download'
end
end
......
......@@ -20,6 +20,7 @@
- [Webhooks](web_hooks/web_hooks.md) Let GitLab notify you when new code has been pushed to your project.
- [Workflow](workflow/README.md) Using GitLab functionality and importing projects from GitHub and SVN.
- [University](university/README.md) Learn Git and GitLab through videos and courses.
- [Git Attributes](user/project/git_attributes.md) Managing Git attributes using a `.gitattributes` file.
## Administrator documentation
......@@ -35,7 +36,7 @@
- [Libravatar](customization/libravatar.md) Use Libravatar instead of Gravatar for user avatars.
- [Log system](administration/logs.md) Log system.
- [Environment Variables](administration/environment_variables.md) to configure GitLab.
- [Operations](operations/README.md) Keeping GitLab up and running.
- [Operations](administration/operations.md) Keeping GitLab up and running.
- [Raketasks](raketasks/README.md) Backups, maintenance, automatic webhook setup and the importing of projects.
- [Repository checks](administration/repository_checks.md) Periodic Git repository checks.
- [Repository storages](administration/repository_storages.md) Manage the paths used to store repositories.
......@@ -47,8 +48,8 @@
- [Migrate GitLab CI to CE/EE](migrate_ci_to_ce/README.md) Follow this guide to migrate your existing GitLab CI data to GitLab CE/EE.
- [Git LFS configuration](workflow/lfs/lfs_administration.md)
- [Housekeeping](administration/housekeeping.md) Keep your Git repository tidy and fast.
- [GitLab Performance Monitoring](monitoring/performance/introduction.md) Configure GitLab and InfluxDB for measuring performance metrics.
- [Monitoring uptime](monitoring/health_check.md) Check the server status using the health check endpoint.
- [GitLab Performance Monitoring](administration/monitoring/performance/introduction.md) Configure GitLab and InfluxDB for measuring performance metrics.
- [Monitoring uptime](user/admin_area/monitoring/health_check.md) Check the server status using the health check endpoint.
- [Debugging Tips](administration/troubleshooting/debug.md) Tips to debug problems when things go wrong
- [Sidekiq Troubleshooting](administration/troubleshooting/sidekiq.md) Debug when Sidekiq appears hung and is not processing jobs.
- [High Availability](administration/high_availability/README.md) Configure multiple servers for scaling or high availability.
......
# GitLab Configuration
GitLab Performance Monitoring is disabled by default. To enable it and change any of its
settings, navigate to the Admin area in **Settings > Metrics**
(`/admin/application_settings`).
The minimum required settings you need to set are the InfluxDB host and port.
Make sure _Enable InfluxDB Metrics_ is checked and hit **Save** to save the
changes.
---
![GitLab Performance Monitoring Admin Settings](img/metrics_gitlab_configuration_settings.png)
---
Finally, a restart of all GitLab processes is required for the changes to take
effect:
```bash
# For Omnibus installations
sudo gitlab-ctl restart
# For installations from source
sudo service gitlab restart
```
## Pending Migrations
When any migrations are pending, the metrics are disabled until the migrations
have been performed.
---
Read more on:
- [Introduction to GitLab Performance Monitoring](introduction.md)
- [InfluxDB Configuration](influxdb_configuration.md)
- [InfluxDB Schema](influxdb_schema.md)
- [Grafana Install/Configuration](grafana_configuration.md)
# Grafana Configuration
[Grafana](http://grafana.org/) is a tool that allows you to visualize time
series metrics through graphs and dashboards. It supports several backend
data stores, including InfluxDB. GitLab writes performance data to InfluxDB
and Grafana will allow you to query InfluxDB to display useful graphs.
For the easiest installation and configuration, install Grafana on the same
server as InfluxDB. For larger installations, you may want to split out these
services.
## Installation
Grafana supplies package repositories (Yum/Apt) for easy installation.
See [Grafana installation documentation](http://docs.grafana.org/installation/)
for detailed steps.
> **Note**: Before starting Grafana for the first time, set the admin user
and password in `/etc/grafana/grafana.ini`. Otherwise, the default password
will be `admin`.
## Configuration
Login as the admin user. Expand the menu by clicking the Grafana logo in the
top left corner. Choose 'Data Sources' from the menu. Then, click 'Add new'
in the top bar.
![Grafana empty data source page](img/grafana_data_source_empty.png)
Fill in the configuration details for the InfluxDB data source. Save and
Test Connection to ensure the configuration is correct.
- **Name**: InfluxDB
- **Default**: Checked
- **Type**: InfluxDB 0.9.x (Even if you're using InfluxDB 0.10.x)
- **Url**: https://localhost:8086 (Or the remote URL if you've installed InfluxDB
on a separate server)
- **Access**: proxy
- **Database**: gitlab
- **User**: admin (Or the username configured when setting up InfluxDB)
- **Password**: The password configured when you set up InfluxDB
![Grafana data source configurations](img/grafana_data_source_configuration.png)
## Apply retention policies and create continuous queries
If you intend to import the GitLab provided Grafana dashboards, you will need to
set up the right retention policies and continuous queries. The easiest way of
doing this is by using the [influxdb-management](https://gitlab.com/gitlab-org/influxdb-management)
repository.
To use this repository you must first clone it:
```
git clone https://gitlab.com/gitlab-org/influxdb-management.git
cd influxdb-management
```
Next you must install the required dependencies:
```
gem install bundler
bundle install
```
Now you must configure the repository by first copying `.env.example` to `.env`
and then editing the `.env` file to contain the correct InfluxDB settings. Once
configured you can simply run `bundle exec rake` and the InfluxDB database will
be configured for you.
For more information see the [influxdb-management README](https://gitlab.com/gitlab-org/influxdb-management/blob/master/README.md).
## Import Dashboards
You can now import a set of default dashboards that will give you a good
start on displaying useful information. GitLab has published a set of default
[Grafana dashboards][grafana-dashboards] to get you started. Clone the
repository or download a zip/tarball, then follow these steps to import each
JSON file.
Open the dashboard dropdown menu and click 'Import'
![Grafana dashboard dropdown](img/grafana_dashboard_dropdown.png)
Click 'Choose file' and browse to the location where you downloaded or cloned
the dashboard repository. Pick one of the JSON files to import.
![Grafana dashboard import](img/grafana_dashboard_import.png)
Once the dashboard is imported, be sure to click save icon in the top bar. If
you do not save the dashboard after importing it will be removed when you
navigate away.
![Grafana save icon](img/grafana_save_icon.png)
Repeat this process for each dashboard you wish to import.
Alternatively you can automatically import all the dashboards into your Grafana
instance. See the README of the [Grafana dashboards][grafana-dashboards]
repository for more information on this process.
[grafana-dashboards]: https://gitlab.com/gitlab-org/grafana-dashboards
---
Read more on:
- [Introduction to GitLab Performance Monitoring](introduction.md)
- [GitLab Configuration](gitlab_configuration.md)
- [InfluxDB Installation/Configuration](influxdb_configuration.md)
- [InfluxDB Schema](influxdb_schema.md)
# InfluxDB Configuration
The default settings provided by [InfluxDB] are not sufficient for a high traffic
GitLab environment. The settings discussed in this document are based on the
settings GitLab uses for GitLab.com, depending on your own needs you may need to
further adjust them.
If you are intending to run InfluxDB on the same server as GitLab, make sure
you have plenty of RAM since InfluxDB can use quite a bit depending on traffic.
Unless you are going with a budget setup, it's advised to run it separately.
## Requirements
- InfluxDB 0.9.5 or newer
- A fairly modern version of Linux
- At least 4GB of RAM
- At least 10GB of storage for InfluxDB data
Note that the RAM and storage requirements can differ greatly depending on the
amount of data received/stored. To limit the amount of stored data users can
look into [InfluxDB Retention Policies][influxdb-retention].
## Installation
Installing InfluxDB is out of the scope of this document. Please refer to the
[InfluxDB documentation].
## InfluxDB Server Settings
Since InfluxDB has many settings that users may wish to customize themselves
(e.g. what port to run InfluxDB on), we'll only cover the essentials.
The configuration file in question is usually located at
`/etc/influxdb/influxdb.conf`. Whenever you make a change in this file,
InfluxDB needs to be restarted.
### Storage Engine
InfluxDB comes with different storage engines and as of InfluxDB 0.9.5 a new
storage engine is available, called [TSM Tree]. All users **must** use the new
`tsm1` storage engine as this [will be the default engine][tsm1-commit] in
upcoming InfluxDB releases.
Make sure you have the following in your configuration file:
```
[data]
dir = "/var/lib/influxdb/data"
engine = "tsm1"
```
### Admin Panel
Production environments should have the InfluxDB admin panel **disabled**. This
feature can be disabled by adding the following to your InfluxDB configuration
file:
```
[admin]
enabled = false
```
### HTTP
HTTP is required when using the [InfluxDB CLI] or other tools such as Grafana,
thus it should be enabled. When enabling make sure to _also_ enable
authentication:
```
[http]
enabled = true
auth-enabled = true
```
_**Note:** Before you enable authentication, you might want to [create an
admin user](#create-a-new-admin-user)._
### UDP
GitLab writes data to InfluxDB via UDP and thus this must be enabled. Enabling
UDP can be done using the following settings:
```
[[udp]]
enabled = true
bind-address = ":8089"
database = "gitlab"
batch-size = 1000
batch-pending = 5
batch-timeout = "1s"
read-buffer = 209715200
```
This does the following:
1. Enable UDP and bind it to port 8089 for all addresses.
2. Store any data received in the "gitlab" database.
3. Define a batch of points to be 1000 points in size and allow a maximum of
5 batches _or_ flush them automatically after 1 second.
4. Define a UDP read buffer size of 200 MB.
One of the most important settings here is the UDP read buffer size as if this
value is set too low, packets will be dropped. You must also make sure the OS
buffer size is set to the same value, the default value is almost never enough.
To set the OS buffer size to 200 MB, on Linux you can run the following command:
```bash
sysctl -w net.core.rmem_max=209715200
```
To make this permanent, add the following to `/etc/sysctl.conf` and restart the
server:
```bash
net.core.rmem_max=209715200
```
It is **very important** to make sure the buffer sizes are large enough to
handle all data sent to InfluxDB as otherwise you _will_ lose data. The above
buffer sizes are based on the traffic for GitLab.com. Depending on the amount of
traffic, users may be able to use a smaller buffer size, but we highly recommend
using _at least_ 100 MB.
When enabling UDP, users should take care to not expose the port to the public,
as doing so will allow anybody to write data into your InfluxDB database (as
[InfluxDB's UDP protocol][udp] doesn't support authentication). We recommend either
whitelisting the allowed IP addresses/ranges, or setting up a VLAN and only
allowing traffic from members of said VLAN.
## Create a new admin user
If you want to [enable authentication](#http), you might want to [create an
admin user][influx-admin]:
```
influx -execute "CREATE USER jeff WITH PASSWORD '1234' WITH ALL PRIVILEGES"
```
## Create the `gitlab` database
Once you get InfluxDB up and running, you need to create a database for GitLab.
Make sure you have changed the [storage engine](#storage-engine) to `tsm1`
before creating a database.
_**Note:** If you [created an admin user](#create-a-new-admin-user) and enabled
[HTTP authentication](#http), remember to append the username (`-username <username>`)
and password (`-password <password>`) you set earlier to the commands below._
Run the following command to create a database named `gitlab`:
```bash
influx -execute 'CREATE DATABASE gitlab'
```
The name **must** be `gitlab`, do not use any other name.
Next, make sure that the database was successfully created:
```bash
influx -execute 'SHOW DATABASES'
```
The output should be similar to:
```
name: databases
---------------
name
_internal
gitlab
```
That's it! Now your GitLab instance should send data to InfluxDB.
---
Read more on:
- [Introduction to GitLab Performance Monitoring](introduction.md)
- [GitLab Configuration](gitlab_configuration.md)
- [InfluxDB Schema](influxdb_schema.md)
- [Grafana Install/Configuration](grafana_configuration.md)
[influxdb-retention]: https://docs.influxdata.com/influxdb/v0.9/query_language/database_management/#retention-policy-management
[influxdb documentation]: https://docs.influxdata.com/influxdb/v0.9/
[influxdb cli]: https://docs.influxdata.com/influxdb/v0.9/tools/shell/
[udp]: https://docs.influxdata.com/influxdb/v0.9/write_protocols/udp/
[influxdb]: https://influxdata.com/time-series-platform/influxdb/
[tsm tree]: https://influxdata.com/blog/new-storage-engine-time-structured-merge-tree/
[tsm1-commit]: https://github.com/influxdata/influxdb/commit/15d723dc77651bac83e09e2b1c94be480966cb0d
[influx-admin]: https://docs.influxdata.com/influxdb/v0.9/administration/authentication_and_authorization/#create-a-new-admin-user
# InfluxDB Schema
The following measurements are currently stored in InfluxDB:
- `PROCESS_file_descriptors`
- `PROCESS_gc_statistics`
- `PROCESS_memory_usage`
- `PROCESS_method_calls`
- `PROCESS_object_counts`
- `PROCESS_transactions`
- `PROCESS_views`
- `events`
Here, `PROCESS` is replaced with either `rails` or `sidekiq` depending on the
process type. In all series, any form of duration is stored in milliseconds.
## PROCESS_file_descriptors
This measurement contains the number of open file descriptors over time. The
value field `value` contains the number of descriptors.
## PROCESS_gc_statistics
This measurement contains Ruby garbage collection statistics such as the amount
of minor/major GC runs (relative to the last sampling interval), the time spent
in garbage collection cycles, and all fields/values returned by `GC.stat`.
## PROCESS_memory_usage
This measurement contains the process' memory usage (in bytes) over time. The
value field `value` contains the number of bytes.
## PROCESS_method_calls
This measurement contains the methods called during a transaction along with
their duration, and a name of the transaction action that invoked the method (if
available). The method call duration is stored in the value field `duration`,
while the method name is stored in the tag `method`. The tag `action` contains
the full name of the transaction action. Both the `method` and `action` fields
are in the following format:
```
ClassName#method_name
```
For example, a method called by the `show` method in the `UsersController` class
would have `action` set to `UsersController#show`.
## PROCESS_object_counts
This measurement is used to store retained Ruby objects (per class) and the
amount of retained objects. The number of objects is stored in the `count` value
field while the class name is stored in the `type` tag.
## PROCESS_transactions
This measurement is used to store basic transaction details such as the time it
took to complete a transaction, how much time was spent in SQL queries, etc. The
following value fields are available:
| Value | Description |
| ----- | ----------- |
| `duration` | The total duration of the transaction |
| `allocated_memory` | The amount of bytes allocated while the transaction was running. This value is only reliable when using single-threaded application servers |
| `method_duration` | The total time spent in method calls |
| `sql_duration` | The total time spent in SQL queries |
| `view_duration` | The total time spent in views |
## PROCESS_views
This measurement is used to store view rendering timings for a transaction. The
following value fields are available:
| Value | Description |
| ----- | ----------- |
| `duration` | The rendering time of the view |
| `view` | The path of the view, relative to the application's root directory |
The `action` tag contains the action name of the transaction that rendered the
view.
## events
This measurement is used to store generic events such as the number of Git
pushes, Emails sent, etc. Each point in this measurement has a single value
field called `count`. The value of this field is simply set to `1`. Each point
also has at least one tag: `event`. This tag's value is set to the event name.
Depending on the event type additional tags may be available as well.
---
Read more on:
- [Introduction to GitLab Performance Monitoring](introduction.md)
- [GitLab Configuration](gitlab_configuration.md)
- [InfluxDB Configuration](influxdb_configuration.md)
- [Grafana Install/Configuration](grafana_configuration.md)
# GitLab Performance Monitoring
GitLab comes with its own application performance measuring system as of GitLab
8.4, simply called "GitLab Performance Monitoring". GitLab Performance Monitoring is available in both the
Community and Enterprise editions.
Apart from this introduction, you are advised to read through the following
documents in order to understand and properly configure GitLab Performance Monitoring:
- [GitLab Configuration](gitlab_configuration.md)
- [InfluxDB Install/Configuration](influxdb_configuration.md)
- [InfluxDB Schema](influxdb_schema.md)
- [Grafana Install/Configuration](grafana_configuration.md)
## Introduction to GitLab Performance Monitoring
GitLab Performance Monitoring makes it possible to measure a wide variety of statistics
including (but not limited to):
- The time it took to complete a transaction (a web request or Sidekiq job).
- The time spent in running SQL queries and rendering HAML views.
- The time spent executing (instrumented) Ruby methods.
- Ruby object allocations, and retained objects in particular.
- System statistics such as the process' memory usage and open file descriptors.
- Ruby garbage collection statistics.
Metrics data is written to [InfluxDB][influxdb] over [UDP][influxdb-udp]. Stored
data can be visualized using [Grafana][grafana] or any other application that
supports reading data from InfluxDB. Alternatively data can be queried using the
InfluxDB CLI.
## Metric Types
Two types of metrics are collected:
1. Transaction specific metrics.
1. Sampled metrics, collected at a certain interval in a separate thread.
### Transaction Metrics
Transaction metrics are metrics that can be associated with a single
transaction. This includes statistics such as the transaction duration, timings
of any executed SQL queries, time spent rendering HAML views, etc. These metrics
are collected for every Rack request and Sidekiq job processed.
### Sampled Metrics
Sampled metrics are metrics that can't be associated with a single transaction.
Examples include garbage collection statistics and retained Ruby objects. These
metrics are collected at a regular interval. This interval is made up out of two
parts:
1. A user defined interval.
1. A randomly generated offset added on top of the interval, the same offset
can't be used twice in a row.
The actual interval can be anywhere between a half of the defined interval and a
half above the interval. For example, for a user defined interval of 15 seconds
the actual interval can be anywhere between 7.5 and 22.5. The interval is
re-generated for every sampling run instead of being generated once and re-used
for the duration of the process' lifetime.
[influxdb]: https://influxdata.com/time-series-platform/influxdb/
[influxdb-udp]: https://docs.influxdata.com/influxdb/v0.9/write_protocols/udp/
[grafana]: http://grafana.org/
# GitLab operations
- [Sidekiq MemoryKiller](operations/sidekiq_memory_killer.md)
- [Cleaning up Redis sessions](operations/cleaning_up_redis_sessions.md)
- [Understanding Unicorn and unicorn-worker-killer](operations/unicorn.md)
- [Moving repositories to a new location](operations/moving_repositories.md)
# Cleaning up stale Redis sessions
Since version 6.2, GitLab stores web user sessions as key-value pairs in Redis.
Prior to GitLab 7.3, user sessions did not automatically expire from Redis. If
you have been running a large GitLab server (thousands of users) since before
GitLab 7.3 we recommend cleaning up stale sessions to compact the Redis
database after you upgrade to GitLab 7.3. You can also perform a cleanup while
still running GitLab 7.2 or older, but in that case new stale sessions will
start building up again after you clean up.
In GitLab versions prior to 7.3.0, the session keys in Redis are 16-byte
hexadecimal values such as '976aa289e2189b17d7ef525a6702ace9'. Starting with
GitLab 7.3.0, the keys are
prefixed with 'session:gitlab:', so they would look like
'session:gitlab:976aa289e2189b17d7ef525a6702ace9'. Below we describe how to
remove the keys in the old format.
First we define a shell function with the proper Redis connection details.
```
rcli() {
# This example works for Omnibus installations of GitLab 7.3 or newer. For an
# installation from source you will have to change the socket path and the
# path to redis-cli.
sudo /opt/gitlab/embedded/bin/redis-cli -s /var/opt/gitlab/redis/redis.socket "$@"
}
# test the new shell function; the response should be PONG
rcli ping
```
Now we do a search to see if there are any session keys in the old format for
us to clean up.
```
# returns the number of old-format session keys in Redis
rcli keys '*' | grep '^[a-f0-9]\{32\}$' | wc -l
```
If the number is larger than zero, you can proceed to expire the keys from
Redis. If the number is zero there is nothing to clean up.
```
# Tell Redis to expire each matched key after 600 seconds.
rcli keys '*' | grep '^[a-f0-9]\{32\}$' | awk '{ print "expire", $0, 600 }' | rcli
# This will print '(integer) 1' for each key that gets expired.
```
Over the next 15 minutes (10 minutes expiry time plus 5 minutes Redis
background save interval) your Redis database will be compacted. If you are
still using GitLab 7.2, users who are not clicking around in GitLab during the
10 minute expiry window will be signed out of GitLab.
# Moving repositories managed by GitLab
Sometimes you need to move all repositories managed by GitLab to
another filesystem or another server. In this document we will look
at some of the ways you can copy all your repositories from
`/var/opt/gitlab/git-data/repositories` to `/mnt/gitlab/repositories`.
We will look at three scenarios: the target directory is empty, the
target directory contains an outdated copy of the repositories, and
how to deal with thousands of repositories.
**Each of the approaches we list can/will overwrite data in the
target directory `/mnt/gitlab/repositories`. Do not mix up the
source and the target.**
## Target directory is empty: use a tar pipe
If the target directory `/mnt/gitlab/repositories` is empty the
simplest thing to do is to use a tar pipe. This method has low
overhead and tar is almost always already installed on your system.
However, it is not possible to resume an interrupted tar pipe: if
that happens then all data must be copied again.
```
# As the git user
tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\
tar -C /mnt/gitlab/repositories -xf -
```
If you want to see progress, replace `-xf` with `-xvf`.
### Tar pipe to another server
You can also use a tar pipe to copy data to another server. If your
'git' user has SSH access to the newserver as 'git@newserver', you
can pipe the data through SSH.
```
# As the git user
tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\
ssh git@newserver tar -C /mnt/gitlab/repositories -xf -
```
If you want to compress the data before it goes over the network
(which will cost you CPU cycles) you can replace `ssh` with `ssh -C`.
## The target directory contains an outdated copy of the repositories: use rsync
If the target directory already contains a partial / outdated copy
of the repositories it may be wasteful to copy all the data again
with tar. In this scenario it is better to use rsync. This utility
is either already installed on your system or easily installable
via apt, yum etc.
```
# As the 'git' user
rsync -a --delete /var/opt/gitlab/git-data/repositories/. \
/mnt/gitlab/repositories
```
The `/.` in the command above is very important, without it you can
easily get the wrong directory structure in the target directory.
If you want to see progress, replace `-a` with `-av`.
### Single rsync to another server
If the 'git' user on your source system has SSH access to the target
server you can send the repositories over the network with rsync.
```
# As the 'git' user
rsync -a --delete /var/opt/gitlab/git-data/repositories/. \
git@newserver:/mnt/gitlab/repositories
```
## Thousands of Git repositories: use one rsync per repository
Every time you start an rsync job it has to inspect all files in
the source directory, all files in the target directory, and then
decide what files to copy or not. If the source or target directory
has many contents this startup phase of rsync can become a burden
for your GitLab server. In cases like this you can make rsync's
life easier by dividing its work in smaller pieces, and sync one
repository at a time.
In addition to rsync we will use [GNU
Parallel](http://www.gnu.org/software/parallel/). This utility is
not included in GitLab so you need to install it yourself with apt
or yum. Also note that the GitLab scripts we used below were added
in GitLab 8.1.
** This process does not clean up repositories at the target location that no
longer exist at the source. ** If you start using your GitLab instance with
`/mnt/gitlab/repositories`, you need to run `gitlab-rake gitlab:cleanup:repos`
after switching to the new repository storage directory.
### Parallel rsync for all repositories known to GitLab
This will sync repositories with 10 rsync processes at a time. We keep
track of progress so that the transfer can be restarted if necessary.
First we create a new directory, owned by 'git', to hold transfer
logs. We assume the directory is empty before we start the transfer
procedure, and that we are the only ones writing files in it.
```
# Omnibus
sudo mkdir /var/opt/gitlab/transfer-logs
sudo chown git:git /var/opt/gitlab/transfer-logs
# Source
sudo -u git -H mkdir /home/git/transfer-logs
```
We seed the process with a list of the directories we want to copy.
```
# Omnibus
sudo -u git sh -c 'gitlab-rake gitlab:list_repos > /var/opt/gitlab/transfer-logs/all-repos-$(date +%s).txt'
# Source
cd /home/git/gitlab
sudo -u git -H sh -c 'bundle exec rake gitlab:list_repos > /home/git/transfer-logs/all-repos-$(date +%s).txt'
```
Now we can start the transfer. The command below is idempotent, and
the number of jobs done by GNU Parallel should converge to zero. If it
does not some repositories listed in all-repos-1234.txt may have been
deleted/renamed before they could be copied.
```
# Omnibus
sudo -u git sh -c '
cat /var/opt/gitlab/transfer-logs/* | sort | uniq -u |\
/usr/bin/env JOBS=10 \
/opt/gitlab/embedded/service/gitlab-rails/bin/parallel-rsync-repos \
/var/opt/gitlab/transfer-logs/success-$(date +%s).log \
/var/opt/gitlab/git-data/repositories \
/mnt/gitlab/repositories
'
# Source
cd /home/git/gitlab
sudo -u git -H sh -c '
cat /home/git/transfer-logs/* | sort | uniq -u |\
/usr/bin/env JOBS=10 \
bin/parallel-rsync-repos \
/home/git/transfer-logs/success-$(date +%s).log \
/home/git/repositories \
/mnt/gitlab/repositories
`
```
### Parallel rsync only for repositories with recent activity
Suppose you have already done one sync that started after 2015-10-1 12:00 UTC.
Then you might only want to sync repositories that were changed via GitLab
_after_ that time. You can use the 'SINCE' variable to tell 'rake
gitlab:list_repos' to only print repositories with recent activity.
```
# Omnibus
sudo gitlab-rake gitlab:list_repos SINCE='2015-10-1 12:00 UTC' |\
sudo -u git \
/usr/bin/env JOBS=10 \
/opt/gitlab/embedded/service/gitlab-rails/bin/parallel-rsync-repos \
success-$(date +%s).log \
/var/opt/gitlab/git-data/repositories \
/mnt/gitlab/repositories
# Source
cd /home/git/gitlab
sudo -u git -H bundle exec rake gitlab:list_repos SINCE='2015-10-1 12:00 UTC' |\
sudo -u git -H \
/usr/bin/env JOBS=10 \
bin/parallel-rsync-repos \
success-$(date +%s).log \
/home/git/repositories \
/mnt/gitlab/repositories
```
# Sidekiq MemoryKiller
The GitLab Rails application code suffers from memory leaks. For web requests
this problem is made manageable using
[unicorn-worker-killer](https://github.com/kzk/unicorn-worker-killer) which
restarts Unicorn worker processes in between requests when needed. The Sidekiq
MemoryKiller applies the same approach to the Sidekiq processes used by GitLab
to process background jobs.
Unlike unicorn-worker-killer, which is enabled by default for all GitLab
installations since GitLab 6.4, the Sidekiq MemoryKiller is enabled by default
_only_ for Omnibus packages. The reason for this is that the MemoryKiller
relies on Runit to restart Sidekiq after a memory-induced shutdown and GitLab
installations from source do not all use Runit or an equivalent.
With the default settings, the MemoryKiller will cause a Sidekiq restart no
more often than once every 15 minutes, with the restart causing about one
minute of delay for incoming background jobs.
## Configuring the MemoryKiller
The MemoryKiller is controlled using environment variables.
- `SIDEKIQ_MEMORY_KILLER_MAX_RSS`: if this variable is set, and its value is
greater than 0, then after each Sidekiq job, the MemoryKiller will check the
RSS of the Sidekiq process that executed the job. If the RSS of the Sidekiq
process (expressed in kilobytes) exceeds SIDEKIQ_MEMORY_KILLER_MAX_RSS, a
delayed shutdown is triggered. The default value for Omnibus packages is set
[in the omnibus-gitlab
repository](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/attributes/default.rb).
- `SIDEKIQ_MEMORY_KILLER_GRACE_TIME`: defaults 900 seconds (15 minutes). When
a shutdown is triggered, the Sidekiq process will keep working normally for
another 15 minutes.
- `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_WAIT`: defaults to 30 seconds. When the grace
time has expired, the MemoryKiller tells Sidekiq to stop accepting new jobs.
Existing jobs get 30 seconds to finish. After that, the MemoryKiller tells
Sidekiq to shut down, and an external supervision mechanism (e.g. Runit) must
restart Sidekiq.
- `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_SIGNAL`: defaults to `SIGKILL`. The name of
the final signal sent to the Sidekiq process when we want it to shut down.
# Understanding Unicorn and unicorn-worker-killer
## Unicorn
GitLab uses [Unicorn](http://unicorn.bogomips.org/), a pre-forking Ruby web
server, to handle web requests (web browsers and Git HTTP clients). Unicorn is
a daemon written in Ruby and C that can load and run a Ruby on Rails
application; in our case the Rails application is GitLab Community Edition or
GitLab Enterprise Edition.
Unicorn has a multi-process architecture to make better use of available CPU
cores (processes can run on different cores) and to have stronger fault
tolerance (most failures stay isolated in only one process and cannot take down
GitLab entirely). On startup, the Unicorn 'master' process loads a clean Ruby
environment with the GitLab application code, and then spawns 'workers' which
inherit this clean initial environment. The 'master' never handles any
requests, that is left to the workers. The operating system network stack
queues incoming requests and distributes them among the workers.
In a perfect world, the master would spawn its pool of workers once, and then
the workers handle incoming web requests one after another until the end of
time. In reality, worker processes can crash or time out: if the master notices
that a worker takes too long to handle a request it will terminate the worker
process with SIGKILL ('kill -9'). No matter how the worker process ended, the
master process will replace it with a new 'clean' process again. Unicorn is
designed to be able to replace 'crashed' workers without dropping user
requests.
This is what a Unicorn worker timeout looks like in `unicorn_stderr.log`. The
master process has PID 56227 below.
```
[2015-06-05T10:58:08.660325 #56227] ERROR -- : worker=10 PID:53009 timeout (61s > 60s), killing
[2015-06-05T10:58:08.699360 #56227] ERROR -- : reaped #<Process::Status: pid 53009 SIGKILL (signal 9)> worker=10
[2015-06-05T10:58:08.708141 #62538] INFO -- : worker=10 spawned pid=62538
[2015-06-05T10:58:08.708824 #62538] INFO -- : worker=10 ready
```
### Tunables
The main tunables for Unicorn are the number of worker processes and the
request timeout after which the Unicorn master terminates a worker process.
See the [omnibus-gitlab Unicorn settings
documentation](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/unicorn.md)
if you want to adjust these settings.
## unicorn-worker-killer
GitLab has memory leaks. These memory leaks manifest themselves in long-running
processes, such as Unicorn workers. (The Unicorn master process is not known to
leak memory, probably because it does not handle user requests.)
To make these memory leaks manageable, GitLab comes with the
[unicorn-worker-killer gem](https://github.com/kzk/unicorn-worker-killer). This
gem [monkey-patches](https://en.wikipedia.org/wiki/Monkey_patch) the Unicorn
workers to do a memory self-check after every 16 requests. If the memory of the
Unicorn worker exceeds a pre-set limit then the worker process exits. The
Unicorn master then automatically replaces the worker process.
This is a robust way to handle memory leaks: Unicorn is designed to handle
workers that 'crash' so no user requests will be dropped. The
unicorn-worker-killer gem is designed to only terminate a worker process _in
between requests_, so no user requests are affected.
This is what a Unicorn worker memory restart looks like in unicorn_stderr.log.
You see that worker 4 (PID 125918) is inspecting itself and decides to exit.
The threshold memory value was 254802235 bytes, about 250MB. With GitLab this
threshold is a random value between 200 and 250 MB. The master process (PID
117565) then reaps the worker process and spawns a new 'worker 4' with PID
127549.
```
[2015-06-05T12:07:41.828374 #125918] WARN -- : #<Unicorn::HttpServer:0x00000002734770>: worker (pid: 125918) exceeds memory limit (256413696 bytes > 254802235 bytes)
[2015-06-05T12:07:41.828472 #125918] WARN -- : Unicorn::WorkerKiller send SIGQUIT (pid: 125918) alive: 23 sec (trial 1)
[2015-06-05T12:07:42.025916 #117565] INFO -- : reaped #<Process::Status: pid 125918 exit 0> worker=4
[2015-06-05T12:07:42.034527 #127549] INFO -- : worker=4 spawned pid=127549
[2015-06-05T12:07:42.035217 #127549] INFO -- : worker=4 ready
```
One other thing that stands out in the log snippet above, taken from
GitLab.com, is that 'worker 4' was serving requests for only 23 seconds. This
is a normal value for our current GitLab.com setup and traffic.
The high frequency of Unicorn memory restarts on some GitLab sites can be a
source of confusion for administrators. Usually they are a [red
herring](https://en.wikipedia.org/wiki/Red_herring).
......@@ -17,6 +17,8 @@ following locations:
- [Commits](commits.md)
- [Deployments](deployments.md)
- [Deploy Keys](deploy_keys.md)
- [Gitignores templates](templates/gitignores.md)
- [GitLab CI Config templates](templates/gitlab_ci_ymls.md)
- [Groups](groups.md)
- [Group Access Requests](access_requests.md)
- [Group Members](members.md)
......@@ -25,7 +27,7 @@ following locations:
- [Labels](labels.md)
- [Merge Requests](merge_requests.md)
- [Milestones](milestones.md)
- [Open source license templates](licenses.md)
- [Open source license templates](templates/licenses.md)
- [Namespaces](namespaces.md)
- [Notes](notes.md) (comments)
- [Notification settings](notification_settings.md)
......
......@@ -436,7 +436,7 @@ Parameters:
### Get project events
Get the events for the specified project.
Sorted from newest to latest
Sorted from newest to oldest
```
GET /projects/:id/events
......
# Gitignores
## List gitignore templates
Get all gitignore templates.
```
GET /templates/gitignores
```
```bash
curl https://gitlab.example.com/api/v3/templates/gitignores
```
Example response:
```json
[
{
"name": "AppEngine"
},
{
"name": "Laravel"
},
{
"name": "Elisp"
},
{
"name": "SketchUp"
},
{
"name": "Ada"
},
{
"name": "Ruby"
},
{
"name": "Kohana"
},
{
"name": "Nanoc"
},
{
"name": "Erlang"
},
{
"name": "OCaml"
},
{
"name": "Lithium"
},
{
"name": "Fortran"
},
{
"name": "Scala"
},
{
"name": "Node"
},
{
"name": "Fancy"
},
{
"name": "Perl"
},
{
"name": "Zephir"
},
{
"name": "WordPress"
},
{
"name": "Symfony"
},
{
"name": "FuelPHP"
},
{
"name": "DM"
},
{
"name": "Sdcc"
},
{
"name": "Rust"
},
{
"name": "C"
},
{
"name": "Umbraco"
},
{
"name": "Actionscript"
},
{
"name": "Android"
},
{
"name": "Grails"
},
{
"name": "Composer"
},
{
"name": "ExpressionEngine"
},
{
"name": "Gcov"
},
{
"name": "Qt"
},
{
"name": "Phalcon"
},
{
"name": "ArchLinuxPackages"
},
{
"name": "TeX"
},
{
"name": "SCons"
},
{
"name": "Lilypond"
},
{
"name": "CommonLisp"
},
{
"name": "Rails"
},
{
"name": "Mercury"
},
{
"name": "Magento"
},
{
"name": "ChefCookbook"
},
{
"name": "GitBook"
},
{
"name": "C++"
},
{
"name": "Eagle"
},
{
"name": "Go"
},
{
"name": "OpenCart"
},
{
"name": "Scheme"
},
{
"name": "Typo3"
},
{
"name": "SeamGen"
},
{
"name": "Swift"
},
{
"name": "Elm"
},
{
"name": "Unity"
},
{
"name": "Agda"
},
{
"name": "CUDA"
},
{
"name": "VVVV"
},
{
"name": "Finale"
},
{
"name": "LemonStand"
},
{
"name": "Textpattern"
},
{
"name": "Julia"
},
{
"name": "Packer"
},
{
"name": "Scrivener"
},
{
"name": "Dart"
},
{
"name": "Plone"
},
{
"name": "Jekyll"
},
{
"name": "Xojo"
},
{
"name": "LabVIEW"
},
{
"name": "Autotools"
},
{
"name": "KiCad"
},
{
"name": "Prestashop"
},
{
"name": "ROS"
},
{
"name": "Smalltalk"
},
{
"name": "GWT"
},
{
"name": "OracleForms"
},
{
"name": "SugarCRM"
},
{
"name": "Nim"
},
{
"name": "SymphonyCMS"
},
{
"name": "Maven"
},
{
"name": "CFWheels"
},
{
"name": "Python"
},
{
"name": "ZendFramework"
},
{
"name": "CakePHP"
},
{
"name": "Concrete5"
},
{
"name": "PlayFramework"
},
{
"name": "Terraform"
},
{
"name": "Elixir"
},
{
"name": "CMake"
},
{
"name": "Joomla"
},
{
"name": "Coq"
},
{
"name": "Delphi"
},
{
"name": "Haskell"
},
{
"name": "Yii"
},
{
"name": "Java"
},
{
"name": "UnrealEngine"
},
{
"name": "AppceleratorTitanium"
},
{
"name": "CraftCMS"
},
{
"name": "ForceDotCom"
},
{
"name": "ExtJs"
},
{
"name": "MetaProgrammingSystem"
},
{
"name": "D"
},
{
"name": "Objective-C"
},
{
"name": "RhodesRhomobile"
},
{
"name": "R"
},
{
"name": "EPiServer"
},
{
"name": "Yeoman"
},
{
"name": "VisualStudio"
},
{
"name": "Processing"
},
{
"name": "Leiningen"
},
{
"name": "Stella"
},
{
"name": "Opa"
},
{
"name": "Drupal"
},
{
"name": "TurboGears2"
},
{
"name": "Idris"
},
{
"name": "Jboss"
},
{
"name": "CodeIgniter"
},
{
"name": "Qooxdoo"
},
{
"name": "Waf"
},
{
"name": "Sass"
},
{
"name": "Lua"
},
{
"name": "Clojure"
},
{
"name": "IGORPro"
},
{
"name": "Gradle"
},
{
"name": "Archives"
},
{
"name": "SynopsysVCS"
},
{
"name": "Ninja"
},
{
"name": "Tags"
},
{
"name": "OSX"
},
{
"name": "Dreamweaver"
},
{
"name": "CodeKit"
},
{
"name": "NotepadPP"
},
{
"name": "VisualStudioCode"
},
{
"name": "Mercurial"
},
{
"name": "BricxCC"
},
{
"name": "DartEditor"
},
{
"name": "Eclipse"
},
{
"name": "Cloud9"
},
{
"name": "TortoiseGit"
},
{
"name": "NetBeans"
},
{
"name": "GPG"
},
{
"name": "Espresso"
},
{
"name": "Redcar"
},
{
"name": "Xcode"
},
{
"name": "Matlab"
},
{
"name": "LyX"
},
{
"name": "SlickEdit"
},
{
"name": "Dropbox"
},
{
"name": "CVS"
},
{
"name": "Calabash"
},
{
"name": "JDeveloper"
},
{
"name": "Vagrant"
},
{
"name": "IPythonNotebook"
},
{
"name": "TextMate"
},
{
"name": "Ensime"
},
{
"name": "WebMethods"
},
{
"name": "VirtualEnv"
},
{
"name": "Emacs"
},
{
"name": "Momentics"
},
{
"name": "JetBrains"
},
{
"name": "SublimeText"
},
{
"name": "Kate"
},
{
"name": "ModelSim"
},
{
"name": "Redis"
},
{
"name": "KDevelop4"
},
{
"name": "Bazaar"
},
{
"name": "Linux"
},
{
"name": "Windows"
},
{
"name": "XilinxISE"
},
{
"name": "Lazarus"
},
{
"name": "EiffelStudio"
},
{
"name": "Anjuta"
},
{
"name": "Vim"
},
{
"name": "Otto"
},
{
"name": "MicrosoftOffice"
},
{
"name": "LibreOffice"
},
{
"name": "SBT"
},
{
"name": "MonoDevelop"
},
{
"name": "SVN"
},
{
"name": "FlexBuilder"
}
]
```
## Single gitignore template
Get a single gitignore template.
```
GET /templates/gitignores/:key
```
| Attribute | Type | Required | Description |
| ---------- | ------ | -------- | ----------- |
| `key` | string | yes | The key of the gitignore template |
```bash
curl https://gitlab.example.com/api/v3/templates/gitignores/Ruby
```
Example response:
```json
{
"name": "Ruby",
"content": "*.gem\n*.rbc\n/.config\n/coverage/\n/InstalledFiles\n/pkg/\n/spec/reports/\n/spec/examples.txt\n/test/tmp/\n/test/version_tmp/\n/tmp/\n\n# Used by dotenv library to load environment variables.\n# .env\n\n## Specific to RubyMotion:\n.dat*\n.repl_history\nbuild/\n*.bridgesupport\nbuild-iPhoneOS/\nbuild-iPhoneSimulator/\n\n## Specific to RubyMotion (use of CocoaPods):\n#\n# We recommend against adding the Pods directory to your .gitignore. However\n# you should judge for yourself, the pros and cons are mentioned at:\n# https://guides.cocoapods.org/using/using-cocoapods.html#should-i-check-the-pods-directory-into-source-control\n#\n# vendor/Pods/\n\n## Documentation cache and generated files:\n/.yardoc/\n/_yardoc/\n/doc/\n/rdoc/\n\n## Environment normalization:\n/.bundle/\n/vendor/bundle\n/lib/bundler/man/\n\n# for a library or gem, you might want to ignore these files since the code is\n# intended to run in multiple environments; otherwise, check them in:\n# Gemfile.lock\n# .ruby-version\n# .ruby-gemset\n\n# unless supporting rvm < 1.11.0 or doing something fancy, ignore this:\n.rvmrc\n"
}
```
# GitLab CI YMLs
## List GitLab CI YML templates
Get all GitLab CI YML templates.
```
GET /templates/gitlab_ci_ymls
```
```bash
curl https://gitlab.example.com/api/v3/templates/gitlab_ci_ymls
```
Example response:
```json
[
{
"name": "C++"
},
{
"name": "Docker"
},
{
"name": "Elixir"
},
{
"name": "LaTeX"
},
{
"name": "Grails"
},
{
"name": "Rust"
},
{
"name": "Nodejs"
},
{
"name": "Ruby"
},
{
"name": "Scala"
},
{
"name": "Maven"
},
{
"name": "Harp"
},
{
"name": "Pelican"
},
{
"name": "Hyde"
},
{
"name": "Nanoc"
},
{
"name": "Octopress"
},
{
"name": "JBake"
},
{
"name": "HTML"
},
{
"name": "Hugo"
},
{
"name": "Metalsmith"
},
{
"name": "Hexo"
},
{
"name": "Lektor"
},
{
"name": "Doxygen"
},
{
"name": "Brunch"
},
{
"name": "Jekyll"
},
{
"name": "Middleman"
}
]
```
## Single GitLab CI YML template
Get a single GitLab CI YML template.
```
GET /templates/gitlab_ci_ymls/:key
```
| Attribute | Type | Required | Description |
| ---------- | ------ | -------- | ----------- |
| `key` | string | yes | The key of the GitLab CI YML template |
```bash
curl https://gitlab.example.com/api/v3/templates/gitlab_ci_ymls/Ruby
```
Example response:
```json
{
"name": "Ruby",
"content": "# This file is a template, and might need editing before it works on your project.\n# Official language image. Look for the different tagged releases at:\n# https://hub.docker.com/r/library/ruby/tags/\nimage: \"ruby:2.3\"\n\n# Pick zero or more services to be used on all builds.\n# Only needed when using a docker container to run your tests in.\n# Check out: http://docs.gitlab.com/ce/ci/docker/using_docker_images.html#what-is-service\nservices:\n - mysql:latest\n - redis:latest\n - postgres:latest\n\nvariables:\n POSTGRES_DB: database_name\n\n# Cache gems in between builds\ncache:\n paths:\n - vendor/ruby\n\n# This is a basic example for a gem or script which doesn't use\n# services such as redis or postgres\nbefore_script:\n - ruby -v # Print out ruby version for debugging\n # Uncomment next line if your rails app needs a JS runtime:\n # - apt-get update -q && apt-get install nodejs -yqq\n - gem install bundler --no-ri --no-rdoc # Bundler is not installed with the image\n - bundle install -j $(nproc) --path vendor # Install dependencies into ./vendor/ruby\n\n# Optional - Delete if not using `rubocop`\nrubocop:\n script:\n - rubocop\n\nrspec:\n script:\n - rspec spec\n\nrails:\n variables:\n DATABASE_URL: \"postgresql://postgres:postgres@postgres:5432/$POSTGRES_DB\"\n script:\n - bundle exec rake db:migrate\n - bundle exec rake db:seed\n - bundle exec rake test\n"
}
```
......@@ -5,7 +5,7 @@
Get all license templates.
```
GET /licenses
GET /templates/licenses
```
| Attribute | Type | Required | Description |
......@@ -13,7 +13,7 @@ GET /licenses
| `popular` | boolean | no | If passed, returns only popular licenses |
```bash
curl https://gitlab.example.com/api/v3/licenses?popular=1
curl https://gitlab.example.com/api/v3/templates/licenses?popular=1
```
Example response:
......@@ -102,7 +102,7 @@ Get a single license template. You can pass parameters to replace the license
placeholder.
```
GET /licenses/:key
GET /templates/licenses/:key
```
| Attribute | Type | Required | Description |
......@@ -116,7 +116,7 @@ If you omit the `fullname` parameter but authenticate your request, the name of
the authenticated user will be used to replace the copyright holder placeholder.
```bash
curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/licenses/mit?project=My+Cool+Project
curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/templates/licenses/mit?project=My+Cool+Project
```
Example response:
......
......@@ -627,3 +627,149 @@ Parameters:
Will return `200 OK` on success, `404 User Not Found` is user cannot be found or
`403 Forbidden` when trying to unblock a user blocked by LDAP synchronization.
### Get user contribution events
Get the contribution events for the specified user, sorted from newest to oldest.
```
GET /users/:id/events
```
Parameters:
| Attribute | Type | Required | Description |
| --------- | ---- | -------- | ----------- |
| `id` | integer | yes | The ID of the user |
```bash
curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/user/:id/events
```
Example response:
```json
[
{
"title": null,
"project_id": 15,
"action_name": "closed",
"target_id": 830,
"target_type": "Issue",
"author_id": 1,
"data": null,
"target_title": "Public project search field",
"author": {
"name": "Dmitriy Zaporozhets",
"username": "root",
"id": 1,
"state": "active",
"avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png",
"web_url": "http://localhost:3000/u/root"
},
"author_username": "root"
},
{
"title": null,
"project_id": 15,
"action_name": "opened",
"target_id": null,
"target_type": null,
"author_id": 1,
"author": {
"name": "Dmitriy Zaporozhets",
"username": "root",
"id": 1,
"state": "active",
"avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png",
"web_url": "http://localhost:3000/u/root"
},
"author_username": "john",
"data": {
"before": "50d4420237a9de7be1304607147aec22e4a14af7",
"after": "c5feabde2d8cd023215af4d2ceeb7a64839fc428",
"ref": "refs/heads/master",
"user_id": 1,
"user_name": "Dmitriy Zaporozhets",
"repository": {
"name": "gitlabhq",
"url": "git@dev.gitlab.org:gitlab/gitlabhq.git",
"description": "GitLab: self hosted Git management software. \r\nDistributed under the MIT License.",
"homepage": "https://dev.gitlab.org/gitlab/gitlabhq"
},
"commits": [
{
"id": "c5feabde2d8cd023215af4d2ceeb7a64839fc428",
"message": "Add simple search to projects in public area",
"timestamp": "2013-05-13T18:18:08+00:00",
"url": "https://dev.gitlab.org/gitlab/gitlabhq/commit/c5feabde2d8cd023215af4d2ceeb7a64839fc428",
"author": {
"name": "Dmitriy Zaporozhets",
"email": "dmitriy.zaporozhets@gmail.com"
}
}
],
"total_commits_count": 1
},
"target_title": null
},
{
"title": null,
"project_id": 15,
"action_name": "closed",
"target_id": 840,
"target_type": "Issue",
"author_id": 1,
"data": null,
"target_title": "Finish & merge Code search PR",
"author": {
"name": "Dmitriy Zaporozhets",
"username": "root",
"id": 1,
"state": "active",
"avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png",
"web_url": "http://localhost:3000/u/root"
},
"author_username": "root"
},
{
"title": null,
"project_id": 15,
"action_name": "commented on",
"target_id": 1312,
"target_type": "Note",
"author_id": 1,
"data": null,
"target_title": null,
"created_at": "2015-12-04T10:33:58.089Z",
"note": {
"id": 1312,
"body": "What an awesome day!",
"attachment": null,
"author": {
"name": "Dmitriy Zaporozhets",
"username": "root",
"id": 1,
"state": "active",
"avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png",
"web_url": "http://localhost:3000/u/root"
},
"created_at": "2015-12-04T10:33:56.698Z",
"system": false,
"upvote": false,
"downvote": false,
"noteable_id": 377,
"noteable_type": "Issue"
},
"author": {
"name": "Dmitriy Zaporozhets",
"username": "root",
"id": 1,
"state": "active",
"avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png",
"web_url": "http://localhost:3000/u/root"
},
"author_username": "root"
}
]
```
......@@ -314,6 +314,29 @@ In this case:
- different highlighting languages are used for each config in the code block
- the [references](#references) guide is used for reconfigure/restart
## Fake tokens
There may be times where a token is needed to demonstrate an API call using
cURL or a secret variable used in CI. It is strongly advised not to use real
tokens in documentation even if the probability of a token being exploited is
low.
You can use the following fake tokens as examples.
| **Token type** | **Token value** |
| --------------------- | --------------------------------- |
| Private user token | `9koXpg98eAheJpvBs5tK` |
| Personal access token | `n671WNGecHugsdEDPsyo` |
| Application ID | `2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6` |
| Application secret | `04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df` |
| Secret CI variable | `Li8j-mLUVA3eZYjPfd_H` |
| Specific Runner token | `yrnZW46BrtBFqM7xDzE7dddd` |
| Shared Runner token | `6Vk7ZsosqQyfreAxXTZr` |
| Trigger token | `be20d8dcc028677c931e04f3871a9b` |
| Webhook secret token | `6XhDroRcYPM5by_h-HLY` |
| Health check token | `Tu7BgjR9qeZTEyRzGG2P` |
| Request profile token | `7VgpS4Ax5utVD2esNstz` |
## API
Here is a list of must-have items. Use them in the exact order that appears
......
# Health Check
> [Introduced][ce-3888] in GitLab 8.8.
GitLab provides a health check endpoint for uptime monitoring on the `health_check` web
endpoint. The health check reports on the overall system status based on the status of
the database connection, the state of the database migrations, and the ability to write
and access the cache. This endpoint can be provided to uptime monitoring services like
[Pingdom][pingdom], [Nagios][nagios-health], and [NewRelic][newrelic-health].
## Access Token
An access token needs to be provided while accessing the health check endpoint. The current
accepted token can be found on the `admin/health_check` page of your GitLab instance.
![access token](img/health_check_token.png)
The access token can be passed as a URL parameter:
```
https://gitlab.example.com/health_check.json?token=ACCESS_TOKEN
```
or as an HTTP header:
```bash
curl --header "TOKEN: ACCESS_TOKEN" https://gitlab.example.com/health_check.json
```
## Using the Endpoint
Once you have the access token, health information can be retrieved as plain text, JSON,
or XML using the `health_check` endpoint:
- `https://gitlab.example.com/health_check?token=ACCESS_TOKEN`
- `https://gitlab.example.com/health_check.json?token=ACCESS_TOKEN`
- `https://gitlab.example.com/health_check.xml?token=ACCESS_TOKEN`
You can also ask for the status of specific services:
- `https://gitlab.example.com/health_check/cache.json?token=ACCESS_TOKEN`
- `https://gitlab.example.com/health_check/database.json?token=ACCESS_TOKEN`
- `https://gitlab.example.com/health_check/migrations.json?token=ACCESS_TOKEN`
For example, the JSON output of the following health check:
```bash
curl --header "TOKEN: ACCESS_TOKEN" https://gitlab.example.com/health_check.json
```
would be like:
```
{"healthy":true,"message":"success"}
```
## Status
On failure, the endpoint will return a `500` HTTP status code. On success, the endpoint
will return a valid successful HTTP status code, and a `success` message. Ideally your
uptime monitoring should look for the success message.
[ce-3888]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3888
[pingdom]: https://www.pingdom.com
[nagios-health]: https://nagios-plugins.org/doc/man/check_http.html
[newrelic-health]: https://docs.newrelic.com/docs/alerts/alert-policies/downtime-alerts/availability-monitoring
This document was moved to [user/admin_area/monitoring/health_check](../user/admin_area/monitoring/health_check.md).
# GitLab Configuration
GitLab Performance Monitoring is disabled by default. To enable it and change any of its
settings, navigate to the Admin area in **Settings > Metrics**
(`/admin/application_settings`).
The minimum required settings you need to set are the InfluxDB host and port.
Make sure _Enable InfluxDB Metrics_ is checked and hit **Save** to save the
changes.
---
![GitLab Performance Monitoring Admin Settings](img/metrics_gitlab_configuration_settings.png)
---
Finally, a restart of all GitLab processes is required for the changes to take
effect:
```bash
# For Omnibus installations
sudo gitlab-ctl restart
# For installations from source
sudo service gitlab restart
```
## Pending Migrations
When any migrations are pending, the metrics are disabled until the migrations
have been performed.
---
Read more on:
- [Introduction to GitLab Performance Monitoring](introduction.md)
- [InfluxDB Configuration](influxdb_configuration.md)
- [InfluxDB Schema](influxdb_schema.md)
- [Grafana Install/Configuration](grafana_configuration.md)
This document was moved to [administration/monitoring/performance/gitlab_configuration](../administration/monitoring/performance/gitlab_configuration.md).
# Grafana Configuration
[Grafana](http://grafana.org/) is a tool that allows you to visualize time
series metrics through graphs and dashboards. It supports several backend
data stores, including InfluxDB. GitLab writes performance data to InfluxDB
and Grafana will allow you to query InfluxDB to display useful graphs.
For the easiest installation and configuration, install Grafana on the same
server as InfluxDB. For larger installations, you may want to split out these
services.
## Installation
Grafana supplies package repositories (Yum/Apt) for easy installation.
See [Grafana installation documentation](http://docs.grafana.org/installation/)
for detailed steps.
> **Note**: Before starting Grafana for the first time, set the admin user
and password in `/etc/grafana/grafana.ini`. Otherwise, the default password
will be `admin`.
## Configuration
Login as the admin user. Expand the menu by clicking the Grafana logo in the
top left corner. Choose 'Data Sources' from the menu. Then, click 'Add new'
in the top bar.
![Grafana empty data source page](img/grafana_data_source_empty.png)
Fill in the configuration details for the InfluxDB data source. Save and
Test Connection to ensure the configuration is correct.
- **Name**: InfluxDB
- **Default**: Checked
- **Type**: InfluxDB 0.9.x (Even if you're using InfluxDB 0.10.x)
- **Url**: https://localhost:8086 (Or the remote URL if you've installed InfluxDB
on a separate server)
- **Access**: proxy
- **Database**: gitlab
- **User**: admin (Or the username configured when setting up InfluxDB)
- **Password**: The password configured when you set up InfluxDB
![Grafana data source configurations](img/grafana_data_source_configuration.png)
## Apply retention policies and create continuous queries
If you intend to import the GitLab provided Grafana dashboards, you will need to
set up the right retention policies and continuous queries. The easiest way of
doing this is by using the [influxdb-management](https://gitlab.com/gitlab-org/influxdb-management)
repository.
To use this repository you must first clone it:
```
git clone https://gitlab.com/gitlab-org/influxdb-management.git
cd influxdb-management
```
Next you must install the required dependencies:
```
gem install bundler
bundle install
```
Now you must configure the repository by first copying `.env.example` to `.env`
and then editing the `.env` file to contain the correct InfluxDB settings. Once
configured you can simply run `bundle exec rake` and the InfluxDB database will
be configured for you.
For more information see the [influxdb-management README](https://gitlab.com/gitlab-org/influxdb-management/blob/master/README.md).
## Import Dashboards
You can now import a set of default dashboards that will give you a good
start on displaying useful information. GitLab has published a set of default
[Grafana dashboards][grafana-dashboards] to get you started. Clone the
repository or download a zip/tarball, then follow these steps to import each
JSON file.
Open the dashboard dropdown menu and click 'Import'
![Grafana dashboard dropdown](img/grafana_dashboard_dropdown.png)
Click 'Choose file' and browse to the location where you downloaded or cloned
the dashboard repository. Pick one of the JSON files to import.
![Grafana dashboard import](img/grafana_dashboard_import.png)
Once the dashboard is imported, be sure to click save icon in the top bar. If
you do not save the dashboard after importing it will be removed when you
navigate away.
![Grafana save icon](img/grafana_save_icon.png)
Repeat this process for each dashboard you wish to import.
Alternatively you can automatically import all the dashboards into your Grafana
instance. See the README of the [Grafana dashboards][grafana-dashboards]
repository for more information on this process.
[grafana-dashboards]: https://gitlab.com/gitlab-org/grafana-dashboards
---
Read more on:
- [Introduction to GitLab Performance Monitoring](introduction.md)
- [GitLab Configuration](gitlab_configuration.md)
- [InfluxDB Installation/Configuration](influxdb_configuration.md)
- [InfluxDB Schema](influxdb_schema.md)
This document was moved to [administration/monitoring/performance/grafana_configuration](../administration/monitoring/performance/grafana_configuration.md).
# InfluxDB Configuration
The default settings provided by [InfluxDB] are not sufficient for a high traffic
GitLab environment. The settings discussed in this document are based on the
settings GitLab uses for GitLab.com, depending on your own needs you may need to
further adjust them.
If you are intending to run InfluxDB on the same server as GitLab, make sure
you have plenty of RAM since InfluxDB can use quite a bit depending on traffic.
Unless you are going with a budget setup, it's advised to run it separately.
## Requirements
- InfluxDB 0.9.5 or newer
- A fairly modern version of Linux
- At least 4GB of RAM
- At least 10GB of storage for InfluxDB data
Note that the RAM and storage requirements can differ greatly depending on the
amount of data received/stored. To limit the amount of stored data users can
look into [InfluxDB Retention Policies][influxdb-retention].
## Installation
Installing InfluxDB is out of the scope of this document. Please refer to the
[InfluxDB documentation].
## InfluxDB Server Settings
Since InfluxDB has many settings that users may wish to customize themselves
(e.g. what port to run InfluxDB on), we'll only cover the essentials.
The configuration file in question is usually located at
`/etc/influxdb/influxdb.conf`. Whenever you make a change in this file,
InfluxDB needs to be restarted.
### Storage Engine
InfluxDB comes with different storage engines and as of InfluxDB 0.9.5 a new
storage engine is available, called [TSM Tree]. All users **must** use the new
`tsm1` storage engine as this [will be the default engine][tsm1-commit] in
upcoming InfluxDB releases.
Make sure you have the following in your configuration file:
```
[data]
dir = "/var/lib/influxdb/data"
engine = "tsm1"
```
### Admin Panel
Production environments should have the InfluxDB admin panel **disabled**. This
feature can be disabled by adding the following to your InfluxDB configuration
file:
```
[admin]
enabled = false
```
### HTTP
HTTP is required when using the [InfluxDB CLI] or other tools such as Grafana,
thus it should be enabled. When enabling make sure to _also_ enable
authentication:
```
[http]
enabled = true
auth-enabled = true
```
_**Note:** Before you enable authentication, you might want to [create an
admin user](#create-a-new-admin-user)._
### UDP
GitLab writes data to InfluxDB via UDP and thus this must be enabled. Enabling
UDP can be done using the following settings:
```
[[udp]]
enabled = true
bind-address = ":8089"
database = "gitlab"
batch-size = 1000
batch-pending = 5
batch-timeout = "1s"
read-buffer = 209715200
```
This does the following:
1. Enable UDP and bind it to port 8089 for all addresses.
2. Store any data received in the "gitlab" database.
3. Define a batch of points to be 1000 points in size and allow a maximum of
5 batches _or_ flush them automatically after 1 second.
4. Define a UDP read buffer size of 200 MB.
One of the most important settings here is the UDP read buffer size as if this
value is set too low, packets will be dropped. You must also make sure the OS
buffer size is set to the same value, the default value is almost never enough.
To set the OS buffer size to 200 MB, on Linux you can run the following command:
```bash
sysctl -w net.core.rmem_max=209715200
```
To make this permanent, add the following to `/etc/sysctl.conf` and restart the
server:
```bash
net.core.rmem_max=209715200
```
It is **very important** to make sure the buffer sizes are large enough to
handle all data sent to InfluxDB as otherwise you _will_ lose data. The above
buffer sizes are based on the traffic for GitLab.com. Depending on the amount of
traffic, users may be able to use a smaller buffer size, but we highly recommend
using _at least_ 100 MB.
When enabling UDP, users should take care to not expose the port to the public,
as doing so will allow anybody to write data into your InfluxDB database (as
[InfluxDB's UDP protocol][udp] doesn't support authentication). We recommend either
whitelisting the allowed IP addresses/ranges, or setting up a VLAN and only
allowing traffic from members of said VLAN.
## Create a new admin user
If you want to [enable authentication](#http), you might want to [create an
admin user][influx-admin]:
```
influx -execute "CREATE USER jeff WITH PASSWORD '1234' WITH ALL PRIVILEGES"
```
## Create the `gitlab` database
Once you get InfluxDB up and running, you need to create a database for GitLab.
Make sure you have changed the [storage engine](#storage-engine) to `tsm1`
before creating a database.
_**Note:** If you [created an admin user](#create-a-new-admin-user) and enabled
[HTTP authentication](#http), remember to append the username (`-username <username>`)
and password (`-password <password>`) you set earlier to the commands below._
Run the following command to create a database named `gitlab`:
```bash
influx -execute 'CREATE DATABASE gitlab'
```
The name **must** be `gitlab`, do not use any other name.
Next, make sure that the database was successfully created:
```bash
influx -execute 'SHOW DATABASES'
```
The output should be similar to:
```
name: databases
---------------
name
_internal
gitlab
```
That's it! Now your GitLab instance should send data to InfluxDB.
---
Read more on:
- [Introduction to GitLab Performance Monitoring](introduction.md)
- [GitLab Configuration](gitlab_configuration.md)
- [InfluxDB Schema](influxdb_schema.md)
- [Grafana Install/Configuration](grafana_configuration.md)
[influxdb-retention]: https://docs.influxdata.com/influxdb/v0.9/query_language/database_management/#retention-policy-management
[influxdb documentation]: https://docs.influxdata.com/influxdb/v0.9/
[influxdb cli]: https://docs.influxdata.com/influxdb/v0.9/tools/shell/
[udp]: https://docs.influxdata.com/influxdb/v0.9/write_protocols/udp/
[influxdb]: https://influxdata.com/time-series-platform/influxdb/
[tsm tree]: https://influxdata.com/blog/new-storage-engine-time-structured-merge-tree/
[tsm1-commit]: https://github.com/influxdata/influxdb/commit/15d723dc77651bac83e09e2b1c94be480966cb0d
[influx-admin]: https://docs.influxdata.com/influxdb/v0.9/administration/authentication_and_authorization/#create-a-new-admin-user
This document was moved to [administration/monitoring/performance/influxdb_configuration](../administration/monitoring/performance/influxdb_configuration.md).
# InfluxDB Schema
The following measurements are currently stored in InfluxDB:
- `PROCESS_file_descriptors`
- `PROCESS_gc_statistics`
- `PROCESS_memory_usage`
- `PROCESS_method_calls`
- `PROCESS_object_counts`
- `PROCESS_transactions`
- `PROCESS_views`
- `events`
Here, `PROCESS` is replaced with either `rails` or `sidekiq` depending on the
process type. In all series, any form of duration is stored in milliseconds.
## PROCESS_file_descriptors
This measurement contains the number of open file descriptors over time. The
value field `value` contains the number of descriptors.
## PROCESS_gc_statistics
This measurement contains Ruby garbage collection statistics such as the amount
of minor/major GC runs (relative to the last sampling interval), the time spent
in garbage collection cycles, and all fields/values returned by `GC.stat`.
## PROCESS_memory_usage
This measurement contains the process' memory usage (in bytes) over time. The
value field `value` contains the number of bytes.
## PROCESS_method_calls
This measurement contains the methods called during a transaction along with
their duration, and a name of the transaction action that invoked the method (if
available). The method call duration is stored in the value field `duration`,
while the method name is stored in the tag `method`. The tag `action` contains
the full name of the transaction action. Both the `method` and `action` fields
are in the following format:
```
ClassName#method_name
```
For example, a method called by the `show` method in the `UsersController` class
would have `action` set to `UsersController#show`.
## PROCESS_object_counts
This measurement is used to store retained Ruby objects (per class) and the
amount of retained objects. The number of objects is stored in the `count` value
field while the class name is stored in the `type` tag.
## PROCESS_transactions
This measurement is used to store basic transaction details such as the time it
took to complete a transaction, how much time was spent in SQL queries, etc. The
following value fields are available:
| Value | Description |
| ----- | ----------- |
| `duration` | The total duration of the transaction |
| `allocated_memory` | The amount of bytes allocated while the transaction was running. This value is only reliable when using single-threaded application servers |
| `method_duration` | The total time spent in method calls |
| `sql_duration` | The total time spent in SQL queries |
| `view_duration` | The total time spent in views |
## PROCESS_views
This measurement is used to store view rendering timings for a transaction. The
following value fields are available:
| Value | Description |
| ----- | ----------- |
| `duration` | The rendering time of the view |
| `view` | The path of the view, relative to the application's root directory |
The `action` tag contains the action name of the transaction that rendered the
view.
## events
This measurement is used to store generic events such as the number of Git
pushes, Emails sent, etc. Each point in this measurement has a single value
field called `count`. The value of this field is simply set to `1`. Each point
also has at least one tag: `event`. This tag's value is set to the event name.
Depending on the event type additional tags may be available as well.
---
Read more on:
- [Introduction to GitLab Performance Monitoring](introduction.md)
- [GitLab Configuration](gitlab_configuration.md)
- [InfluxDB Configuration](influxdb_configuration.md)
- [Grafana Install/Configuration](grafana_configuration.md)
This document was moved to [administration/monitoring/performance/influxdb_schema](../administration/monitoring/performance/influxdb_schema.md).
# GitLab Performance Monitoring
GitLab comes with its own application performance measuring system as of GitLab
8.4, simply called "GitLab Performance Monitoring". GitLab Performance Monitoring is available in both the
Community and Enterprise editions.
Apart from this introduction, you are advised to read through the following
documents in order to understand and properly configure GitLab Performance Monitoring:
- [GitLab Configuration](gitlab_configuration.md)
- [InfluxDB Install/Configuration](influxdb_configuration.md)
- [InfluxDB Schema](influxdb_schema.md)
- [Grafana Install/Configuration](grafana_configuration.md)
## Introduction to GitLab Performance Monitoring
GitLab Performance Monitoring makes it possible to measure a wide variety of statistics
including (but not limited to):
- The time it took to complete a transaction (a web request or Sidekiq job).
- The time spent in running SQL queries and rendering HAML views.
- The time spent executing (instrumented) Ruby methods.
- Ruby object allocations, and retained objects in particular.
- System statistics such as the process' memory usage and open file descriptors.
- Ruby garbage collection statistics.
Metrics data is written to [InfluxDB][influxdb] over [UDP][influxdb-udp]. Stored
data can be visualized using [Grafana][grafana] or any other application that
supports reading data from InfluxDB. Alternatively data can be queried using the
InfluxDB CLI.
## Metric Types
Two types of metrics are collected:
1. Transaction specific metrics.
1. Sampled metrics, collected at a certain interval in a separate thread.
### Transaction Metrics
Transaction metrics are metrics that can be associated with a single
transaction. This includes statistics such as the transaction duration, timings
of any executed SQL queries, time spent rendering HAML views, etc. These metrics
are collected for every Rack request and Sidekiq job processed.
### Sampled Metrics
Sampled metrics are metrics that can't be associated with a single transaction.
Examples include garbage collection statistics and retained Ruby objects. These
metrics are collected at a regular interval. This interval is made up out of two
parts:
1. A user defined interval.
1. A randomly generated offset added on top of the interval, the same offset
can't be used twice in a row.
The actual interval can be anywhere between a half of the defined interval and a
half above the interval. For example, for a user defined interval of 15 seconds
the actual interval can be anywhere between 7.5 and 22.5. The interval is
re-generated for every sampling run instead of being generated once and re-used
for the duration of the process' lifetime.
[influxdb]: https://influxdata.com/time-series-platform/influxdb/
[influxdb-udp]: https://docs.influxdata.com/influxdb/v0.9/write_protocols/udp/
[grafana]: http://grafana.org/
This document was moved to [administration/monitoring/performance/introduction](../administration/monitoring/performance/introduction.md).
# GitLab operations
- [Sidekiq MemoryKiller](sidekiq_memory_killer.md)
- [Cleaning up Redis sessions](cleaning_up_redis_sessions.md)
- [Understanding Unicorn and unicorn-worker-killer](unicorn.md)
This document was moved to [administration/operations](../administration/operations.md).
# Cleaning up stale Redis sessions
Since version 6.2, GitLab stores web user sessions as key-value pairs in Redis.
Prior to GitLab 7.3, user sessions did not automatically expire from Redis. If
you have been running a large GitLab server (thousands of users) since before
GitLab 7.3 we recommend cleaning up stale sessions to compact the Redis
database after you upgrade to GitLab 7.3. You can also perform a cleanup while
still running GitLab 7.2 or older, but in that case new stale sessions will
start building up again after you clean up.
In GitLab versions prior to 7.3.0, the session keys in Redis are 16-byte
hexadecimal values such as '976aa289e2189b17d7ef525a6702ace9'. Starting with
GitLab 7.3.0, the keys are
prefixed with 'session:gitlab:', so they would look like
'session:gitlab:976aa289e2189b17d7ef525a6702ace9'. Below we describe how to
remove the keys in the old format.
First we define a shell function with the proper Redis connection details.
```
rcli() {
# This example works for Omnibus installations of GitLab 7.3 or newer. For an
# installation from source you will have to change the socket path and the
# path to redis-cli.
sudo /opt/gitlab/embedded/bin/redis-cli -s /var/opt/gitlab/redis/redis.socket "$@"
}
# test the new shell function; the response should be PONG
rcli ping
```
Now we do a search to see if there are any session keys in the old format for
us to clean up.
```
# returns the number of old-format session keys in Redis
rcli keys '*' | grep '^[a-f0-9]\{32\}$' | wc -l
```
If the number is larger than zero, you can proceed to expire the keys from
Redis. If the number is zero there is nothing to clean up.
```
# Tell Redis to expire each matched key after 600 seconds.
rcli keys '*' | grep '^[a-f0-9]\{32\}$' | awk '{ print "expire", $0, 600 }' | rcli
# This will print '(integer) 1' for each key that gets expired.
```
Over the next 15 minutes (10 minutes expiry time plus 5 minutes Redis
background save interval) your Redis database will be compacted. If you are
still using GitLab 7.2, users who are not clicking around in GitLab during the
10 minute expiry window will be signed out of GitLab.
This document was moved to [administration/operations/cleaning_up_redis_sessions](../administration/operations/cleaning_up_redis_sessions.md).
# Moving repositories managed by GitLab
Sometimes you need to move all repositories managed by GitLab to
another filesystem or another server. In this document we will look
at some of the ways you can copy all your repositories from
`/var/opt/gitlab/git-data/repositories` to `/mnt/gitlab/repositories`.
We will look at three scenarios: the target directory is empty, the
target directory contains an outdated copy of the repositories, and
how to deal with thousands of repositories.
**Each of the approaches we list can/will overwrite data in the
target directory `/mnt/gitlab/repositories`. Do not mix up the
source and the target.**
## Target directory is empty: use a tar pipe
If the target directory `/mnt/gitlab/repositories` is empty the
simplest thing to do is to use a tar pipe. This method has low
overhead and tar is almost always already installed on your system.
However, it is not possible to resume an interrupted tar pipe: if
that happens then all data must be copied again.
```
# As the git user
tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\
tar -C /mnt/gitlab/repositories -xf -
```
If you want to see progress, replace `-xf` with `-xvf`.
### Tar pipe to another server
You can also use a tar pipe to copy data to another server. If your
'git' user has SSH access to the newserver as 'git@newserver', you
can pipe the data through SSH.
```
# As the git user
tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\
ssh git@newserver tar -C /mnt/gitlab/repositories -xf -
```
If you want to compress the data before it goes over the network
(which will cost you CPU cycles) you can replace `ssh` with `ssh -C`.
## The target directory contains an outdated copy of the repositories: use rsync
If the target directory already contains a partial / outdated copy
of the repositories it may be wasteful to copy all the data again
with tar. In this scenario it is better to use rsync. This utility
is either already installed on your system or easily installable
via apt, yum etc.
```
# As the 'git' user
rsync -a --delete /var/opt/gitlab/git-data/repositories/. \
/mnt/gitlab/repositories
```
The `/.` in the command above is very important, without it you can
easily get the wrong directory structure in the target directory.
If you want to see progress, replace `-a` with `-av`.
### Single rsync to another server
If the 'git' user on your source system has SSH access to the target
server you can send the repositories over the network with rsync.
```
# As the 'git' user
rsync -a --delete /var/opt/gitlab/git-data/repositories/. \
git@newserver:/mnt/gitlab/repositories
```
## Thousands of Git repositories: use one rsync per repository
Every time you start an rsync job it has to inspect all files in
the source directory, all files in the target directory, and then
decide what files to copy or not. If the source or target directory
has many contents this startup phase of rsync can become a burden
for your GitLab server. In cases like this you can make rsync's
life easier by dividing its work in smaller pieces, and sync one
repository at a time.
In addition to rsync we will use [GNU
Parallel](http://www.gnu.org/software/parallel/). This utility is
not included in GitLab so you need to install it yourself with apt
or yum. Also note that the GitLab scripts we used below were added
in GitLab 8.1.
** This process does not clean up repositories at the target location that no
longer exist at the source. ** If you start using your GitLab instance with
`/mnt/gitlab/repositories`, you need to run `gitlab-rake gitlab:cleanup:repos`
after switching to the new repository storage directory.
### Parallel rsync for all repositories known to GitLab
This will sync repositories with 10 rsync processes at a time. We keep
track of progress so that the transfer can be restarted if necessary.
First we create a new directory, owned by 'git', to hold transfer
logs. We assume the directory is empty before we start the transfer
procedure, and that we are the only ones writing files in it.
```
# Omnibus
sudo mkdir /var/opt/gitlab/transfer-logs
sudo chown git:git /var/opt/gitlab/transfer-logs
# Source
sudo -u git -H mkdir /home/git/transfer-logs
```
We seed the process with a list of the directories we want to copy.
```
# Omnibus
sudo -u git sh -c 'gitlab-rake gitlab:list_repos > /var/opt/gitlab/transfer-logs/all-repos-$(date +%s).txt'
# Source
cd /home/git/gitlab
sudo -u git -H sh -c 'bundle exec rake gitlab:list_repos > /home/git/transfer-logs/all-repos-$(date +%s).txt'
```
Now we can start the transfer. The command below is idempotent, and
the number of jobs done by GNU Parallel should converge to zero. If it
does not some repositories listed in all-repos-1234.txt may have been
deleted/renamed before they could be copied.
```
# Omnibus
sudo -u git sh -c '
cat /var/opt/gitlab/transfer-logs/* | sort | uniq -u |\
/usr/bin/env JOBS=10 \
/opt/gitlab/embedded/service/gitlab-rails/bin/parallel-rsync-repos \
/var/opt/gitlab/transfer-logs/success-$(date +%s).log \
/var/opt/gitlab/git-data/repositories \
/mnt/gitlab/repositories
'
# Source
cd /home/git/gitlab
sudo -u git -H sh -c '
cat /home/git/transfer-logs/* | sort | uniq -u |\
/usr/bin/env JOBS=10 \
bin/parallel-rsync-repos \
/home/git/transfer-logs/success-$(date +%s).log \
/home/git/repositories \
/mnt/gitlab/repositories
`
```
### Parallel rsync only for repositories with recent activity
Suppose you have already done one sync that started after 2015-10-1 12:00 UTC.
Then you might only want to sync repositories that were changed via GitLab
_after_ that time. You can use the 'SINCE' variable to tell 'rake
gitlab:list_repos' to only print repositories with recent activity.
```
# Omnibus
sudo gitlab-rake gitlab:list_repos SINCE='2015-10-1 12:00 UTC' |\
sudo -u git \
/usr/bin/env JOBS=10 \
/opt/gitlab/embedded/service/gitlab-rails/bin/parallel-rsync-repos \
success-$(date +%s).log \
/var/opt/gitlab/git-data/repositories \
/mnt/gitlab/repositories
# Source
cd /home/git/gitlab
sudo -u git -H bundle exec rake gitlab:list_repos SINCE='2015-10-1 12:00 UTC' |\
sudo -u git -H \
/usr/bin/env JOBS=10 \
bin/parallel-rsync-repos \
success-$(date +%s).log \
/home/git/repositories \
/mnt/gitlab/repositories
```
This document was moved to [administration/operations/moving_repositories](../administration/operations/moving_repositories.md).
# Sidekiq MemoryKiller
The GitLab Rails application code suffers from memory leaks. For web requests
this problem is made manageable using
[unicorn-worker-killer](https://github.com/kzk/unicorn-worker-killer) which
restarts Unicorn worker processes in between requests when needed. The Sidekiq
MemoryKiller applies the same approach to the Sidekiq processes used by GitLab
to process background jobs.
Unlike unicorn-worker-killer, which is enabled by default for all GitLab
installations since GitLab 6.4, the Sidekiq MemoryKiller is enabled by default
_only_ for Omnibus packages. The reason for this is that the MemoryKiller
relies on Runit to restart Sidekiq after a memory-induced shutdown and GitLab
installations from source do not all use Runit or an equivalent.
With the default settings, the MemoryKiller will cause a Sidekiq restart no
more often than once every 15 minutes, with the restart causing about one
minute of delay for incoming background jobs.
## Configuring the MemoryKiller
The MemoryKiller is controlled using environment variables.
- `SIDEKIQ_MEMORY_KILLER_MAX_RSS`: if this variable is set, and its value is
greater than 0, then after each Sidekiq job, the MemoryKiller will check the
RSS of the Sidekiq process that executed the job. If the RSS of the Sidekiq
process (expressed in kilobytes) exceeds SIDEKIQ_MEMORY_KILLER_MAX_RSS, a
delayed shutdown is triggered. The default value for Omnibus packages is set
[in the omnibus-gitlab
repository](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/attributes/default.rb).
- `SIDEKIQ_MEMORY_KILLER_GRACE_TIME`: defaults 900 seconds (15 minutes). When
a shutdown is triggered, the Sidekiq process will keep working normally for
another 15 minutes.
- `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_WAIT`: defaults to 30 seconds. When the grace
time has expired, the MemoryKiller tells Sidekiq to stop accepting new jobs.
Existing jobs get 30 seconds to finish. After that, the MemoryKiller tells
Sidekiq to shut down, and an external supervision mechanism (e.g. Runit) must
restart Sidekiq.
- `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_SIGNAL`: defaults to `SIGKILL`. The name of
the final signal sent to the Sidekiq process when we want it to shut down.
This document was moved to [administration/operations/sidekiq_memory_killer](../administration/operations/sidekiq_memory_killer.md).
# Understanding Unicorn and unicorn-worker-killer
## Unicorn
GitLab uses [Unicorn](http://unicorn.bogomips.org/), a pre-forking Ruby web
server, to handle web requests (web browsers and Git HTTP clients). Unicorn is
a daemon written in Ruby and C that can load and run a Ruby on Rails
application; in our case the Rails application is GitLab Community Edition or
GitLab Enterprise Edition.
Unicorn has a multi-process architecture to make better use of available CPU
cores (processes can run on different cores) and to have stronger fault
tolerance (most failures stay isolated in only one process and cannot take down
GitLab entirely). On startup, the Unicorn 'master' process loads a clean Ruby
environment with the GitLab application code, and then spawns 'workers' which
inherit this clean initial environment. The 'master' never handles any
requests, that is left to the workers. The operating system network stack
queues incoming requests and distributes them among the workers.
In a perfect world, the master would spawn its pool of workers once, and then
the workers handle incoming web requests one after another until the end of
time. In reality, worker processes can crash or time out: if the master notices
that a worker takes too long to handle a request it will terminate the worker
process with SIGKILL ('kill -9'). No matter how the worker process ended, the
master process will replace it with a new 'clean' process again. Unicorn is
designed to be able to replace 'crashed' workers without dropping user
requests.
This is what a Unicorn worker timeout looks like in `unicorn_stderr.log`. The
master process has PID 56227 below.
```
[2015-06-05T10:58:08.660325 #56227] ERROR -- : worker=10 PID:53009 timeout (61s > 60s), killing
[2015-06-05T10:58:08.699360 #56227] ERROR -- : reaped #<Process::Status: pid 53009 SIGKILL (signal 9)> worker=10
[2015-06-05T10:58:08.708141 #62538] INFO -- : worker=10 spawned pid=62538
[2015-06-05T10:58:08.708824 #62538] INFO -- : worker=10 ready
```
### Tunables
The main tunables for Unicorn are the number of worker processes and the
request timeout after which the Unicorn master terminates a worker process.
See the [omnibus-gitlab Unicorn settings
documentation](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/unicorn.md)
if you want to adjust these settings.
## unicorn-worker-killer
GitLab has memory leaks. These memory leaks manifest themselves in long-running
processes, such as Unicorn workers. (The Unicorn master process is not known to
leak memory, probably because it does not handle user requests.)
To make these memory leaks manageable, GitLab comes with the
[unicorn-worker-killer gem](https://github.com/kzk/unicorn-worker-killer). This
gem [monkey-patches](https://en.wikipedia.org/wiki/Monkey_patch) the Unicorn
workers to do a memory self-check after every 16 requests. If the memory of the
Unicorn worker exceeds a pre-set limit then the worker process exits. The
Unicorn master then automatically replaces the worker process.
This is a robust way to handle memory leaks: Unicorn is designed to handle
workers that 'crash' so no user requests will be dropped. The
unicorn-worker-killer gem is designed to only terminate a worker process _in
between requests_, so no user requests are affected.
This is what a Unicorn worker memory restart looks like in unicorn_stderr.log.
You see that worker 4 (PID 125918) is inspecting itself and decides to exit.
The threshold memory value was 254802235 bytes, about 250MB. With GitLab this
threshold is a random value between 200 and 250 MB. The master process (PID
117565) then reaps the worker process and spawns a new 'worker 4' with PID
127549.
```
[2015-06-05T12:07:41.828374 #125918] WARN -- : #<Unicorn::HttpServer:0x00000002734770>: worker (pid: 125918) exceeds memory limit (256413696 bytes > 254802235 bytes)
[2015-06-05T12:07:41.828472 #125918] WARN -- : Unicorn::WorkerKiller send SIGQUIT (pid: 125918) alive: 23 sec (trial 1)
[2015-06-05T12:07:42.025916 #117565] INFO -- : reaped #<Process::Status: pid 125918 exit 0> worker=4
[2015-06-05T12:07:42.034527 #127549] INFO -- : worker=4 spawned pid=127549
[2015-06-05T12:07:42.035217 #127549] INFO -- : worker=4 ready
```
One other thing that stands out in the log snippet above, taken from
GitLab.com, is that 'worker 4' was serving requests for only 23 seconds. This
is a normal value for our current GitLab.com setup and traffic.
The high frequency of Unicorn memory restarts on some GitLab sites can be a
source of confusion for administrators. Usually they are a [red
herring](https://en.wikipedia.org/wiki/Red_herring).
This document was moved to [administration/operations/unicorn](../administration/operations/unicorn.md).
# Health Check
> [Introduced][ce-3888] in GitLab 8.8.
GitLab provides a health check endpoint for uptime monitoring on the `health_check` web
endpoint. The health check reports on the overall system status based on the status of
the database connection, the state of the database migrations, and the ability to write
and access the cache. This endpoint can be provided to uptime monitoring services like
[Pingdom][pingdom], [Nagios][nagios-health], and [NewRelic][newrelic-health].
## Access Token
An access token needs to be provided while accessing the health check endpoint. The current
accepted token can be found on the `admin/health_check` page of your GitLab instance.
![access token](img/health_check_token.png)
The access token can be passed as a URL parameter:
```
https://gitlab.example.com/health_check.json?token=ACCESS_TOKEN
```
or as an HTTP header:
```bash
curl --header "TOKEN: ACCESS_TOKEN" https://gitlab.example.com/health_check.json
```
## Using the Endpoint
Once you have the access token, health information can be retrieved as plain text, JSON,
or XML using the `health_check` endpoint:
- `https://gitlab.example.com/health_check?token=ACCESS_TOKEN`
- `https://gitlab.example.com/health_check.json?token=ACCESS_TOKEN`
- `https://gitlab.example.com/health_check.xml?token=ACCESS_TOKEN`
You can also ask for the status of specific services:
- `https://gitlab.example.com/health_check/cache.json?token=ACCESS_TOKEN`
- `https://gitlab.example.com/health_check/database.json?token=ACCESS_TOKEN`
- `https://gitlab.example.com/health_check/migrations.json?token=ACCESS_TOKEN`
For example, the JSON output of the following health check:
```bash
curl --header "TOKEN: ACCESS_TOKEN" https://gitlab.example.com/health_check.json
```
would be like:
```
{"healthy":true,"message":"success"}
```
## Status
On failure, the endpoint will return a `500` HTTP status code. On success, the endpoint
will return a valid successful HTTP status code, and a `success` message. Ideally your
uptime monitoring should look for the success message.
[ce-3888]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3888
[pingdom]: https://www.pingdom.com
[nagios-health]: https://nagios-plugins.org/doc/man/check_http.html
[newrelic-health]: https://docs.newrelic.com/docs/alerts/alert-policies/downtime-alerts/availability-monitoring
......@@ -32,6 +32,7 @@ The following table depicts the various user permission levels in a project.
| See a commit status | | ✓ | ✓ | ✓ | ✓ |
| See a container registry | | ✓ | ✓ | ✓ | ✓ |
| See environments | | ✓ | ✓ | ✓ | ✓ |
| See a list of merge requests | | ✓ | ✓ | ✓ | ✓ |
| Manage/Accept merge requests | | | ✓ | ✓ | ✓ |
| Create new merge request | | | ✓ | ✓ | ✓ |
| Create new branches | | | ✓ | ✓ | ✓ |
......
# Git Attributes
GitLab supports defining custom [Git attributes][gitattributes] such as what
files to treat as binary, and what language to use for syntax highlighting
diffs.
To define these attributes, create a file called `.gitattributes` in the root
directory of your repository and push it to the default branch of your project.
## Encoding Requirements
The `.gitattributes` file _must_ be encoded in UTF-8 and _must not_ contain a
Byte Order Mark. If a different encoding is used, the file's contents will be
ignored.
## Syntax Highlighting
The `.gitattributes` file can be used to define which language to use when
syntax highlighting files and diffs. See ["Syntax
Highlighting"](highlighting.md) for more information.
[gitattributes]: https://git-scm.com/docs/gitattributes
......@@ -46,7 +46,6 @@ module API
mount ::API::Boards
mount ::API::Keys
mount ::API::Labels
mount ::API::LicenseTemplates
mount ::API::Lint
mount ::API::Members
mount ::API::MergeRequests
......
module API
# License Templates API
class LicenseTemplates < Grape::API
PROJECT_TEMPLATE_REGEX =
/[\<\{\[]
(project|description|
one\sline\s.+\swhat\sit\sdoes\.) # matching the start and end is enough here
[\>\}\]]/xi.freeze
YEAR_TEMPLATE_REGEX = /[<{\[](year|yyyy)[>}\]]/i.freeze
FULLNAME_TEMPLATE_REGEX =
/[\<\{\[]
(fullname|name\sof\s(author|copyright\sowner))
[\>\}\]]/xi.freeze
# Get the list of the available license templates
#
# Parameters:
# popular - Filter licenses to only the popular ones
#
# Example Request:
# GET /licenses
# GET /licenses?popular=1
get 'licenses' do
options = {
featured: params[:popular].present? ? true : nil
}
present Licensee::License.all(options), with: Entities::RepoLicense
end
# Get text for specific license
#
# Parameters:
# key (required) - The key of a license
# project - Copyrighted project name
# fullname - Full name of copyright holder
#
# Example Request:
# GET /licenses/mit
#
get 'licenses/:key', requirements: { key: /[\w\.-]+/ } do
required_attributes! [:key]
not_found!('License') unless Licensee::License.find(params[:key])
# We create a fresh Licensee::License object since we'll modify its
# content in place below.
license = Licensee::License.new(params[:key])
license.content.gsub!(YEAR_TEMPLATE_REGEX, Time.now.year.to_s)
license.content.gsub!(PROJECT_TEMPLATE_REGEX, params[:project]) if params[:project].present?
fullname = params[:fullname].presence || current_user.try(:name)
license.content.gsub!(FULLNAME_TEMPLATE_REGEX, fullname) if fullname
present license, with: Entities::RepoLicense
end
end
end
......@@ -416,6 +416,12 @@ module API
required_attributes! [:group_id, :group_access]
attrs = attributes_for_keys [:group_id, :group_access, :expires_at]
group = Group.find_by_id(attrs[:group_id])
unless group && can?(current_user, :read_group, group)
not_found!('Group')
end
unless user_project.allowed_to_share_with_group?
return render_api_error!("The project sharing with group is disabled", 400)
end
......
module API
class Templates < Grape::API
GLOBAL_TEMPLATE_TYPES = {
gitignores: Gitlab::Template::GitignoreTemplate,
gitlab_ci_ymls: Gitlab::Template::GitlabCiYmlTemplate
gitignores: {
klass: Gitlab::Template::GitignoreTemplate,
gitlab_version: 8.8
},
gitlab_ci_ymls: {
klass: Gitlab::Template::GitlabCiYmlTemplate,
gitlab_version: 8.9
}
}.freeze
PROJECT_TEMPLATE_REGEX =
/[\<\{\[]
(project|description|
one\sline\s.+\swhat\sit\sdoes\.) # matching the start and end is enough here
[\>\}\]]/xi.freeze
YEAR_TEMPLATE_REGEX = /[<{\[](year|yyyy)[>}\]]/i.freeze
FULLNAME_TEMPLATE_REGEX =
/[\<\{\[]
(fullname|name\sof\s(author|copyright\sowner))
[\>\}\]]/xi.freeze
DEPRECATION_MESSAGE = ' This endpoint is deprecated and will be removed in GitLab 9.0.'.freeze
helpers do
def parsed_license_template
# We create a fresh Licensee::License object since we'll modify its
# content in place below.
template = Licensee::License.new(params[:name])
template.content.gsub!(YEAR_TEMPLATE_REGEX, Time.now.year.to_s)
template.content.gsub!(PROJECT_TEMPLATE_REGEX, params[:project]) if params[:project].present?
fullname = params[:fullname].presence || current_user.try(:name)
template.content.gsub!(FULLNAME_TEMPLATE_REGEX, fullname) if fullname
template
end
def render_response(template_type, template)
not_found!(template_type.to_s.singularize) unless template
present template, with: Entities::Template
end
end
GLOBAL_TEMPLATE_TYPES.each do |template_type, klass|
# Get the list of the available template
#
# Example Request:
# GET /gitignores
# GET /gitlab_ci_ymls
get template_type.to_s do
{ "licenses" => :deprecated, "templates/licenses" => :ok }.each do |route, status|
desc 'Get the list of the available license template' do
detailed_desc = 'This feature was introduced in GitLab 8.7.'
detailed_desc << DEPRECATION_MESSAGE unless status == :ok
detail detailed_desc
success Entities::RepoLicense
end
params do
optional :popular, type: Boolean, desc: 'If passed, returns only popular licenses'
end
get route do
options = {
featured: declared(params).popular.present? ? true : nil
}
present Licensee::License.all(options), with: Entities::RepoLicense
end
end
{ "licenses/:name" => :deprecated, "templates/licenses/:name" => :ok }.each do |route, status|
desc 'Get the text for a specific license' do
detailed_desc = 'This feature was introduced in GitLab 8.7.'
detailed_desc << DEPRECATION_MESSAGE unless status == :ok
detail detailed_desc
success Entities::RepoLicense
end
params do
requires :name, type: String, desc: 'The name of the template'
end
get route, requirements: { name: /[\w\.-]+/ } do
not_found!('License') unless Licensee::License.find(declared(params).name)
template = parsed_license_template
present template, with: Entities::RepoLicense
end
end
GLOBAL_TEMPLATE_TYPES.each do |template_type, properties|
klass = properties[:klass]
gitlab_version = properties[:gitlab_version]
{ template_type => :deprecated, "templates/#{template_type}" => :ok }.each do |route, status|
desc 'Get the list of the available template' do
detailed_desc = "This feature was introduced in GitLab #{gitlab_version}."
detailed_desc << DEPRECATION_MESSAGE unless status == :ok
detail detailed_desc
success Entities::TemplatesList
end
get route do
present klass.all, with: Entities::TemplatesList
end
end
{ "#{template_type}/:name" => :deprecated, "templates/#{template_type}/:name" => :ok }.each do |route, status|
desc 'Get the text for a specific template present in local filesystem' do
detailed_desc = "This feature was introduced in GitLab #{gitlab_version}."
detailed_desc << DEPRECATION_MESSAGE unless status == :ok
detail detailed_desc
success Entities::Template
end
params do
requires :name, type: String, desc: 'The name of the template'
end
get route do
new_template = klass.find(declared(params).name)
# Get the text for a specific template present in local filesystem
#
# Parameters:
# name (required) - The name of a template
#
# Example Request:
# GET /gitignores/Elixir
# GET /gitlab_ci_ymls/Ruby
get "#{template_type}/:name" do
required_attributes! [:name]
new_template = klass.find(params[:name])
render_response(template_type, new_template)
end
end
end
end
end
......@@ -321,6 +321,26 @@ module API
user.activate
end
end
desc 'Get contribution events of a specified user' do
detail 'This feature was introduced in GitLab 8.13.'
success Entities::Event
end
params do
requires :id, type: String, desc: 'The user ID'
end
get ':id/events' do
user = User.find_by(id: declared(params).id)
not_found!('User') unless user
events = user.recent_events.
merge(ProjectsFinder.new.execute(current_user)).
references(:project).
with_associations.
page(params[:page])
present paginate(events), with: Entities::Event
end
end
resource :user do
......
......@@ -52,8 +52,7 @@ module ExtractsPath
# Append a trailing slash if we only get a ref and no file path
id += '/' unless id.ends_with?('/')
valid_refs = @project.repository.ref_names
valid_refs.select! { |v| id.start_with?("#{v}/") }
valid_refs = ref_names.select { |v| id.start_with?("#{v}/") }
if valid_refs.length == 0
# No exact ref match, so just try our best
......@@ -74,6 +73,19 @@ module ExtractsPath
pair
end
# If we have an ID of 'foo.atom', and the controller provides Atom and HTML
# formats, then we have to check if the request was for the Atom version of
# the ID without the '.atom' suffix, or the HTML version of the ID including
# the suffix. We only check this if the version including the suffix doesn't
# match, so it is possible to create a branch which has an unroutable Atom
# feed.
def extract_ref_without_atom(id)
id_without_atom = id.sub(/\.atom$/, '')
valid_refs = ref_names.select { |v| "#{id_without_atom}/".start_with?("#{v}/") }
valid_refs.max_by(&:length)
end
# Assigns common instance variables for views working with Git tree-ish objects
#
# Assignments are:
......@@ -86,6 +98,10 @@ module ExtractsPath
# If the :id parameter appears to be requesting a specific response format,
# that will be handled as well.
#
# If there is no path and the ref doesn't exist in the repo, try to resolve
# the ref without an '.atom' suffix. If _that_ ref is found, set the request's
# format to Atom manually.
#
# Automatically renders `not_found!` if a valid tree path could not be
# resolved (e.g., when a user inserts an invalid path or ref).
def assign_ref_vars
......@@ -103,6 +119,13 @@ module ExtractsPath
@commit = @repo.commit(@options[:extended_sha1])
end
if @path.empty? && !@commit
@id = @ref = extract_ref_without_atom(@id)
@commit = @repo.commit(@ref)
request.format = :atom if @commit
end
raise InvalidPathError unless @commit
@hex_path = Digest::SHA1.hexdigest(@path)
......@@ -125,4 +148,10 @@ module ExtractsPath
id += "/" + params[:path] unless params[:path].blank?
id
end
def ref_names
return [] unless @project
@ref_names ||= @project.repository.ref_names
end
end
......@@ -102,15 +102,16 @@ describe Projects::CommitController do
describe "as patch" do
include_examples "export as", :patch
let(:format) { :patch }
let(:commit2) { project.commit('498214de67004b1da3d820901307bed2a68a8ef6') }
it "is a git email patch" do
go(id: commit.id, format: format)
go(id: commit2.id, format: format)
expect(response.body).to start_with("From #{commit.id}")
expect(response.body).to start_with("From #{commit2.id}")
end
it "contains a git diff" do
go(id: commit.id, format: format)
go(id: commit2.id, format: format)
expect(response.body).to match(/^diff --git/)
end
......@@ -135,6 +136,8 @@ describe Projects::CommitController do
describe "GET branches" do
it "contains branch and tags information" do
commit = project.commit('5937ac0a7beb003549fc5fd26fc247adbce4a52e')
get(:branches,
namespace_id: project.namespace.to_param,
project_id: project.to_param,
......@@ -254,16 +257,17 @@ describe Projects::CommitController do
end
let(:existing_path) { '.gitmodules' }
let(:commit2) { project.commit('5937ac0a7beb003549fc5fd26fc247adbce4a52e') }
context 'when the commit exists' do
context 'when the user has access to the project' do
context 'when the path exists in the diff' do
it 'enables diff notes' do
diff_for_path(id: commit.id, old_path: existing_path, new_path: existing_path)
diff_for_path(id: commit2.id, old_path: existing_path, new_path: existing_path)
expect(assigns(:diff_notes_disabled)).to be_falsey
expect(assigns(:comments_target)).to eq(noteable_type: 'Commit',
commit_id: commit.id)
commit_id: commit2.id)
end
it 'only renders the diffs for the path given' do
......@@ -272,7 +276,7 @@ describe Projects::CommitController do
meth.call(diffs)
end
diff_for_path(id: commit.id, old_path: existing_path, new_path: existing_path)
diff_for_path(id: commit2.id, old_path: existing_path, new_path: existing_path)
end
end
......
......@@ -10,16 +10,39 @@ describe Projects::CommitsController do
end
describe "GET show" do
context "as atom feed" do
context "when the ref name ends in .atom" do
render_views
context "when the ref does not exist with the suffix" do
it "renders as atom" do
get(:show,
namespace_id: project.namespace.to_param,
project_id: project.to_param,
id: "master",
format: "atom")
id: "master.atom")
expect(response).to be_success
expect(response.content_type).to eq('application/atom+xml')
end
end
context "when the ref exists with the suffix" do
before do
commit = project.repository.commit('master')
allow_any_instance_of(Repository).to receive(:commit).and_call_original
allow_any_instance_of(Repository).to receive(:commit).with('master.atom').and_return(commit)
get(:show,
namespace_id: project.namespace.to_param,
project_id: project.to_param,
id: "master.atom")
end
it "renders as HTML" do
expect(response).to be_success
expect(response.content_type).to eq('text/html')
end
end
end
end
end
......@@ -708,4 +708,52 @@ describe Projects::MergeRequestsController do
end
end
end
describe 'POST assign_related_issues' do
let(:issue1) { create(:issue, project: project) }
let(:issue2) { create(:issue, project: project) }
def post_assign_issues
merge_request.update!(description: "Closes #{issue1.to_reference} and #{issue2.to_reference}",
author: user,
source_branch: 'feature',
target_branch: 'master')
post :assign_related_issues,
namespace_id: project.namespace.to_param,
project_id: project.to_param,
id: merge_request.iid
end
it 'shows a flash message on success' do
post_assign_issues
expect(flash[:notice]).to eq '2 issues have been assigned to you'
end
it 'correctly pluralizes flash message on success' do
issue2.update!(assignee: user)
post_assign_issues
expect(flash[:notice]).to eq '1 issue has been assigned to you'
end
it 'calls MergeRequests::AssignIssuesService' do
expect(MergeRequests::AssignIssuesService).to receive(:new).
with(project, user, merge_request: merge_request).
and_return(double(execute: { count: 1 }))
post_assign_issues
end
it 'is skipped when not signed in' do
project.update!(visibility_level: Gitlab::VisibilityLevel::PUBLIC)
sign_out(:user)
expect(MergeRequests::AssignIssuesService).not_to receive(:new)
post_assign_issues
end
end
end
......@@ -17,4 +17,18 @@ describe Projects::TagsController do
expect(assigns(:releases)).not_to include(invalid_release)
end
end
describe 'GET show' do
before { get :show, namespace_id: project.namespace.to_param, project_id: project.to_param, id: id }
context "valid tag" do
let(:id) { 'v1.0.0' }
it { is_expected.to respond_with(:success) }
end
context "invalid tag" do
let(:id) { 'latest' }
it { is_expected.to respond_with(:not_found) }
end
end
end
require 'rails_helper'
feature 'Merge request issue assignment', js: true, feature: true do
let(:user) { create(:user) }
let(:project) { create(:project, :public) }
let(:issue1) { create(:issue, project: project) }
let(:issue2) { create(:issue, project: project) }
let(:merge_request) { create(:merge_request, :simple, source_project: project, author: user, description: "fixes #{issue1.to_reference} and #{issue2.to_reference}") }
let(:service) { MergeRequests::AssignIssuesService.new(merge_request, user, user, project) }
before do
project.team << [user, :developer]
end
def visit_merge_request(current_user = nil)
login_as(current_user || user)
visit namespace_project_merge_request_path(project.namespace, project, merge_request)
end
context 'logged in as author' do
scenario 'updates related issues' do
visit_merge_request
click_link "Assign yourself to these issues"
expect(page).to have_content "2 issues have been assigned to you"
end
it 'returns user to the merge request' do
visit_merge_request
click_link "Assign yourself to these issues"
expect(page).to have_content merge_request.description
end
it "doesn't display if related issues are already assigned" do
[issue1, issue2].each { |issue| issue.update!(assignee: user) }
visit_merge_request
expect(page).not_to have_content "Assign yourself"
end
end
context 'not MR author' do
it "doesn't not show assignment link" do
visit_merge_request(create(:user))
expect(page).not_to have_content "Assign yourself"
end
end
end
require 'spec_helper'
describe "Guest navigation menu" do
let(:project) { create :empty_project, :private }
let(:guest) { create :user }
before do
project.team << [guest, :guest]
login_as(guest)
end
it "shows allowed tabs only" do
visit namespace_project_path(project.namespace, project)
within(".nav-links") do
expect(page).to have_content 'Project'
expect(page).to have_content 'Activity'
expect(page).to have_content 'Issues'
expect(page).to have_content 'Wiki'
expect(page).not_to have_content 'Repository'
expect(page).not_to have_content 'Pipelines'
expect(page).not_to have_content 'Graphs'
expect(page).not_to have_content 'Merge Requests'
end
end
end
......@@ -203,7 +203,7 @@ describe "Private Project Access", feature: true do
it { is_expected.to be_allowed_for master }
it { is_expected.to be_allowed_for developer }
it { is_expected.to be_allowed_for reporter }
it { is_expected.to be_allowed_for guest }
it { is_expected.to be_denied_for guest }
it { is_expected.to be_denied_for :user }
it { is_expected.to be_denied_for :external }
it { is_expected.to be_denied_for :visitor }
......
......@@ -19,7 +19,7 @@ describe SearchHelper do
expect(subject.filename).to eq('CHANGELOG')
expect(subject.basename).to eq('CHANGELOG')
expect(subject.ref).to eq('master')
expect(subject.startline).to eq(186)
expect(subject.startline).to eq(188)
expect(subject.data.lines[2]).to eq(" - Feature: Replace teams with group membership\n")
end
......
......@@ -6,6 +6,7 @@ describe ExtractsPath, lib: true do
include Gitlab::Routing.url_helpers
let(:project) { double('project') }
let(:request) { double('request') }
before do
@project = project
......@@ -15,9 +16,10 @@ describe ExtractsPath, lib: true do
allow(project).to receive(:repository).and_return(repo)
allow(project).to receive(:path_with_namespace).
and_return('gitlab/gitlab-ci')
allow(request).to receive(:format=)
end
describe '#assign_ref' do
describe '#assign_ref_vars' do
let(:ref) { sample_commit[:id] }
let(:params) { { path: sample_commit[:line_code_path], ref: ref } }
......@@ -61,6 +63,75 @@ describe ExtractsPath, lib: true do
expect(@id).to eq(get_id)
end
end
context 'ref only exists without .atom suffix' do
context 'with a path' do
let(:params) { { ref: 'v1.0.0.atom', path: 'README.md' } }
it 'renders a 404' do
expect(self).to receive(:render_404)
assign_ref_vars
end
end
context 'without a path' do
let(:params) { { ref: 'v1.0.0.atom' } }
before { assign_ref_vars }
it 'sets the un-suffixed version as @ref' do
expect(@ref).to eq('v1.0.0')
end
it 'sets the request format to Atom' do
expect(request).to have_received(:format=).with(:atom)
end
end
end
context 'ref exists with .atom suffix' do
context 'with a path' do
let(:params) { { ref: 'master.atom', path: 'README.md' } }
before do
repository = @project.repository
allow(repository).to receive(:commit).and_call_original
allow(repository).to receive(:commit).with('master.atom').and_return(repository.commit('master'))
assign_ref_vars
end
it 'sets the suffixed version as @ref' do
expect(@ref).to eq('master.atom')
end
it 'does not change the request format' do
expect(request).not_to have_received(:format=)
end
end
context 'without a path' do
let(:params) { { ref: 'master.atom' } }
before do
repository = @project.repository
allow(repository).to receive(:commit).and_call_original
allow(repository).to receive(:commit).with('master.atom').and_return(repository.commit('master'))
end
it 'sets the suffixed version as @ref' do
assign_ref_vars
expect(@ref).to eq('master.atom')
end
it 'does not change the request format' do
expect(request).not_to receive(:format=)
assign_ref_vars
end
end
end
end
describe '#extract_ref' do
......@@ -115,4 +186,18 @@ describe ExtractsPath, lib: true do
end
end
end
describe '#extract_ref_without_atom' do
it 'ignores any matching refs suffixed with atom' do
expect(extract_ref_without_atom('master.atom')).to eq('master')
end
it 'returns the longest matching ref' do
expect(extract_ref_without_atom('release/app/v1.0.0.atom')).to eq('release/app/v1.0.0')
end
it 'returns nil if there are no matching refs' do
expect(extract_ref_without_atom('foo.atom')).to eq(nil)
end
end
end
......@@ -8,13 +8,13 @@ describe Gitlab::DataBuilder::Push, lib: true do
let(:data) { described_class.build_sample(project, user) }
it { expect(data).to be_a(Hash) }
it { expect(data[:before]).to eq('6f6d7e7ed97bb5f0054f2b1df789b39ca89b6ff9') }
it { expect(data[:after]).to eq('5937ac0a7beb003549fc5fd26fc247adbce4a52e') }
it { expect(data[:before]).to eq('1b12f15a11fc6e62177bef08f47bc7b5ce50b141') }
it { expect(data[:after]).to eq('b83d6e391c22777fca1ed3012fce84f633d7fed0') }
it { expect(data[:ref]).to eq('refs/heads/master') }
it { expect(data[:commits].size).to eq(3) }
it { expect(data[:total_commits_count]).to eq(3) }
it { expect(data[:commits].first[:added]).to eq(['gitlab-grack']) }
it { expect(data[:commits].first[:modified]).to eq(['.gitmodules']) }
it { expect(data[:commits].first[:added]).to eq(['bar/branch-test.txt']) }
it { expect(data[:commits].first[:modified]).to eq([]) }
it { expect(data[:commits].first[:removed]).to eq([]) }
include_examples 'project hook data with deprecateds'
......
......@@ -628,7 +628,7 @@ describe Notify do
it_behaves_like 'a user cannot unsubscribe through footer link'
it 'has the correct subject' do
is_expected.to have_subject /#{commit.title} \(#{commit.short_id}\)/
is_expected.to have_subject /Re: #{project.name} | #{commit.title} \(#{commit.short_id}\)/
end
it 'contains a link to the commit' do
......
......@@ -164,10 +164,10 @@ eos
let(:data) { commit.hook_attrs(with_changed_files: true) }
it { expect(data).to be_a(Hash) }
it { expect(data[:message]).to include('Add submodule from gitlab.com') }
it { expect(data[:timestamp]).to eq('2014-02-27T11:01:38+02:00') }
it { expect(data[:added]).to eq(["gitlab-grack"]) }
it { expect(data[:modified]).to eq([".gitmodules"]) }
it { expect(data[:message]).to include('adds bar folder and branch-test text file to check Repository merged_to_root_ref method') }
it { expect(data[:timestamp]).to eq('2016-09-27T14:37:46+00:00') }
it { expect(data[:added]).to eq(["bar/branch-test.txt"]) }
it { expect(data[:modified]).to eq([]) }
it { expect(data[:removed]).to eq([]) }
end
......
......@@ -135,6 +135,17 @@ describe Event, models: true do
it { expect(event.visible_to_user?(member)).to eq true }
it { expect(event.visible_to_user?(guest)).to eq true }
it { expect(event.visible_to_user?(admin)).to eq true }
context 'private project' do
let(:project) { create(:project, :private) }
it { expect(event.visible_to_user?(non_member)).to eq false }
it { expect(event.visible_to_user?(author)).to eq true }
it { expect(event.visible_to_user?(assignee)).to eq true }
it { expect(event.visible_to_user?(member)).to eq true }
it { expect(event.visible_to_user?(guest)).to eq false }
it { expect(event.visible_to_user?(admin)).to eq true }
end
end
end
......
......@@ -5,9 +5,6 @@ describe Key, models: true do
it { is_expected.to belong_to(:user) }
end
describe "Mass assignment" do
end
describe "Validation" do
it { is_expected.to validate_presence_of(:title) }
it { is_expected.to validate_presence_of(:key) }
......
require 'spec_helper'
describe LabelLink, models: true do
let(:label) { create(:label_link) }
it { expect(label).to be_valid }
it { expect(build(:label_link)).to be_valid }
it { is_expected.to belong_to(:label) }
it { is_expected.to belong_to(:target) }
......
......@@ -6,9 +6,9 @@ describe MergeRequestDiff, models: true do
it { expect(subject).to be_valid }
it { expect(subject).to be_persisted }
it { expect(subject.commits.count).to eq(5) }
it { expect(subject.diffs.count).to eq(8) }
it { expect(subject.head_commit_sha).to eq('5937ac0a7beb003549fc5fd26fc247adbce4a52e') }
it { expect(subject.commits.count).to eq(29) }
it { expect(subject.diffs.count).to eq(20) }
it { expect(subject.head_commit_sha).to eq('b83d6e391c22777fca1ed3012fce84f633d7fed0') }
it { expect(subject.base_commit_sha).to eq('ae73cb07c9eeaf35924a10f713b364d32b2dd34f') }
it { expect(subject.start_commit_sha).to eq('0b4bc9a49b562e85de7cc9e834518ea6828729b9') }
end
......
......@@ -489,7 +489,7 @@ describe MergeRequest, models: true do
subject(:merge_request_with_divergence) { create(:merge_request, :diverged, source_project: project, target_project: project) }
it 'counts commits that are on target branch but not on source branch' do
expect(subject.diverged_commits_count).to eq(5)
expect(subject.diverged_commits_count).to eq(29)
end
end
......@@ -497,7 +497,7 @@ describe MergeRequest, models: true do
subject(:merge_request_fork_with_divergence) { create(:merge_request, :diverged, source_project: fork_project, target_project: project) }
it 'counts commits that are on target branch but not on source branch' do
expect(subject.diverged_commits_count).to eq(5)
expect(subject.diverged_commits_count).to eq(29)
end
end
......
......@@ -11,7 +11,7 @@ describe ProjectGroupLink do
it { should validate_presence_of(:project_id) }
it { should validate_uniqueness_of(:group_id).scoped_to(:project_id).with_message(/already shared/) }
it { should validate_presence_of(:group_id) }
it { should validate_presence_of(:group) }
it { should validate_presence_of(:group_access) }
end
end
......@@ -122,11 +122,30 @@ describe Repository, models: true do
end
describe '#merged_to_root_ref?' do
context 'merged branch' do
context 'merged branch without ff' do
subject { repository.merged_to_root_ref?('branch-merged') }
it { is_expected.to be_truthy }
end
# If the HEAD was ff then it will be false
context 'merged with ff' do
subject { repository.merged_to_root_ref?('improve/awesome') }
it { is_expected.to be_truthy }
end
context 'not merged branch' do
subject { repository.merged_to_root_ref?('not-merged-branch') }
it { is_expected.to be_falsey }
end
context 'default branch' do
subject { repository.merged_to_root_ref?('master') }
it { is_expected.to be_falsey }
end
end
describe '#can_be_merged?' do
......@@ -324,7 +343,7 @@ describe Repository, models: true do
subject { results.first }
it { is_expected.to be_an String }
it { expect(subject.lines[2]).to eq("master:CHANGELOG:188: - Feature: Replace teams with group membership\n") }
it { expect(subject.lines[2]).to eq("master:CHANGELOG:190: - Feature: Replace teams with group membership\n") }
end
end
......@@ -968,10 +987,10 @@ describe Repository, models: true do
context 'cherry-picking a merge commit' do
it 'cherry-picks the changes' do
expect(repository.blob_at_branch('master', 'foo/bar/.gitkeep')).to be_nil
expect(repository.blob_at_branch('improve/awesome', 'foo/bar/.gitkeep')).to be_nil
repository.cherry_pick(user, pickable_merge, 'master')
expect(repository.blob_at_branch('master', 'foo/bar/.gitkeep')).not_to be_nil
repository.cherry_pick(user, pickable_merge, 'improve/awesome')
expect(repository.blob_at_branch('improve/awesome', 'foo/bar/.gitkeep')).not_to be_nil
end
end
end
......
......@@ -12,7 +12,7 @@ describe ProjectPolicy, models: true do
[
:read_project, :read_board, :read_list, :read_wiki, :read_issue, :read_label,
:read_milestone, :read_project_snippet, :read_project_member,
:read_merge_request, :read_note, :create_project, :create_issue, :create_note,
:read_note, :create_project, :create_issue, :create_note,
:upload_file
]
end
......@@ -21,7 +21,8 @@ describe ProjectPolicy, models: true do
[
:download_code, :fork_project, :create_project_snippet, :update_issue,
:admin_issue, :admin_label, :admin_list, :read_commit_status, :read_build,
:read_container_image, :read_pipeline, :read_environment, :read_deployment
:read_container_image, :read_pipeline, :read_environment, :read_deployment,
:read_merge_request
]
end
......
......@@ -53,7 +53,12 @@ describe API::API, api: true do
get api("/projects/#{project.id}/repository/commits?until=#{before.utc.iso8601}", user)
if commits.size >= 20
expect(json_response.size).to eq(20)
else
expect(json_response.size).to eq(commits.size - 1)
end
expect(json_response.first["id"]).to eq(commits.second.id)
expect(json_response.second["id"]).to eq(commits.third.id)
end
......@@ -447,11 +452,12 @@ describe API::API, api: true do
end
it 'returns the inline comment' do
post api("/projects/#{project.id}/repository/commits/#{project.repository.commit.id}/comments", user), note: 'My comment', path: project.repository.commit.raw_diffs.first.new_path, line: 7, line_type: 'new'
post api("/projects/#{project.id}/repository/commits/#{project.repository.commit.id}/comments", user), note: 'My comment', path: project.repository.commit.raw_diffs.first.new_path, line: 1, line_type: 'new'
expect(response).to have_http_status(201)
expect(json_response['note']).to eq('My comment')
expect(json_response['path']).to eq(project.repository.commit.raw_diffs.first.new_path)
expect(json_response['line']).to eq(7)
expect(json_response['line']).to eq(1)
expect(json_response['line_type']).to eq('new')
end
......
require 'spec_helper'
describe API::API, api: true do
include ApiHelpers
describe 'Entity' do
before { get api('/licenses/mit') }
it { expect(json_response['key']).to eq('mit') }
it { expect(json_response['name']).to eq('MIT License') }
it { expect(json_response['nickname']).to be_nil }
it { expect(json_response['popular']).to be true }
it { expect(json_response['html_url']).to eq('http://choosealicense.com/licenses/mit/') }
it { expect(json_response['source_url']).to eq('https://opensource.org/licenses/MIT') }
it { expect(json_response['description']).to include('A permissive license that is short and to the point.') }
it { expect(json_response['conditions']).to eq(%w[include-copyright]) }
it { expect(json_response['permissions']).to eq(%w[commercial-use modifications distribution private-use]) }
it { expect(json_response['limitations']).to eq(%w[no-liability]) }
it { expect(json_response['content']).to include('The MIT License (MIT)') }
end
describe 'GET /licenses' do
it 'returns a list of available license templates' do
get api('/licenses')
expect(response).to have_http_status(200)
expect(json_response).to be_an Array
expect(json_response.size).to eq(15)
expect(json_response.map { |l| l['key'] }).to include('agpl-3.0')
end
describe 'the popular parameter' do
context 'with popular=1' do
it 'returns a list of available popular license templates' do
get api('/licenses?popular=1')
expect(response).to have_http_status(200)
expect(json_response).to be_an Array
expect(json_response.size).to eq(3)
expect(json_response.map { |l| l['key'] }).to include('apache-2.0')
end
end
end
end
describe 'GET /licenses/:key' do
context 'with :project and :fullname given' do
before do
get api("/licenses/#{license_type}?project=My+Awesome+Project&fullname=Anton+#{license_type.upcase}")
end
context 'for the mit license' do
let(:license_type) { 'mit' }
it 'returns the license text' do
expect(json_response['content']).to include('The MIT License (MIT)')
end
it 'replaces placeholder values' do
expect(json_response['content']).to include("Copyright (c) #{Time.now.year} Anton")
end
end
context 'for the agpl-3.0 license' do
let(:license_type) { 'agpl-3.0' }
it 'returns the license text' do
expect(json_response['content']).to include('GNU AFFERO GENERAL PUBLIC LICENSE')
end
it 'replaces placeholder values' do
expect(json_response['content']).to include('My Awesome Project')
expect(json_response['content']).to include("Copyright (C) #{Time.now.year} Anton")
end
end
context 'for the gpl-3.0 license' do
let(:license_type) { 'gpl-3.0' }
it 'returns the license text' do
expect(json_response['content']).to include('GNU GENERAL PUBLIC LICENSE')
end
it 'replaces placeholder values' do
expect(json_response['content']).to include('My Awesome Project')
expect(json_response['content']).to include("Copyright (C) #{Time.now.year} Anton")
end
end
context 'for the gpl-2.0 license' do
let(:license_type) { 'gpl-2.0' }
it 'returns the license text' do
expect(json_response['content']).to include('GNU GENERAL PUBLIC LICENSE')
end
it 'replaces placeholder values' do
expect(json_response['content']).to include('My Awesome Project')
expect(json_response['content']).to include("Copyright (C) #{Time.now.year} Anton")
end
end
context 'for the apache-2.0 license' do
let(:license_type) { 'apache-2.0' }
it 'returns the license text' do
expect(json_response['content']).to include('Apache License')
end
it 'replaces placeholder values' do
expect(json_response['content']).to include("Copyright #{Time.now.year} Anton")
end
end
context 'for an uknown license' do
let(:license_type) { 'muth-over9000' }
it 'returns a 404' do
expect(response).to have_http_status(404)
end
end
end
context 'with no :fullname given' do
context 'with an authenticated user' do
let(:user) { create(:user) }
it 'replaces the copyright owner placeholder with the name of the current user' do
get api('/licenses/mit', user)
expect(json_response['content']).to include("Copyright (c) #{Time.now.year} #{user.name}")
end
end
end
end
end
......@@ -588,37 +588,39 @@ describe API::API, api: true do
before do
note = create(:note_on_issue, note: 'What an awesome day!', project: project)
EventCreateService.new.leave_note(note, note.author)
get api("/projects/#{project.id}/events", user)
end
it { expect(response).to have_http_status(200) }
it 'returns all events' do
get api("/projects/#{project.id}/events", user)
context 'joined event' do
let(:json_event) { json_response[1] }
expect(response).to have_http_status(200)
it { expect(json_event['action_name']).to eq('joined') }
it { expect(json_event['project_id'].to_i).to eq(project.id) }
it { expect(json_event['author_username']).to eq(user3.username) }
it { expect(json_event['author']['name']).to eq(user3.name) }
end
first_event = json_response.first
context 'comment event' do
let(:json_event) { json_response.first }
expect(first_event['action_name']).to eq('commented on')
expect(first_event['note']['body']).to eq('What an awesome day!')
it { expect(json_event['action_name']).to eq('commented on') }
it { expect(json_event['note']['body']).to eq('What an awesome day!') }
last_event = json_response.last
expect(last_event['action_name']).to eq('joined')
expect(last_event['project_id'].to_i).to eq(project.id)
expect(last_event['author_username']).to eq(user3.username)
expect(last_event['author']['name']).to eq(user3.name)
end
end
it 'returns a 404 error if not found' do
get api('/projects/42/events', user)
expect(response).to have_http_status(404)
expect(json_response['message']).to eq('404 Project Not Found')
end
it 'returns a 404 error if user is not a member' do
other_user = create(:user)
get api("/projects/#{project.id}/events", other_user)
expect(response).to have_http_status(404)
end
end
......@@ -819,6 +821,20 @@ describe API::API, api: true do
expect(response.status).to eq 400
end
it 'returns a 404 error when user cannot read group' do
private_group = create(:group, :private)
post api("/projects/#{project.id}/share", user), group_id: private_group.id, group_access: Gitlab::Access::DEVELOPER
expect(response.status).to eq 404
end
it 'returns a 404 error when group does not exist' do
post api("/projects/#{project.id}/share", user), group_id: 1234, group_access: Gitlab::Access::DEVELOPER
expect(response.status).to eq 404
end
it "returns a 409 error when wrong params passed" do
post api("/projects/#{project.id}/share", user), group_id: group.id, group_access: 1234
expect(response.status).to eq 409
......
......@@ -21,7 +21,7 @@ describe API::API, api: true do
expect(response).to have_http_status(200)
expect(json_response).to be_an Array
expect(json_response.first['name']).to eq('encoding')
expect(json_response.first['name']).to eq('bar')
expect(json_response.first['type']).to eq('tree')
expect(json_response.first['mode']).to eq('040000')
end
......@@ -166,9 +166,9 @@ describe API::API, api: true do
expect(response).to have_http_status(200)
expect(json_response).to be_an Array
contributor = json_response.first
expect(contributor['email']).to eq('dmitriy.zaporozhets@gmail.com')
expect(contributor['name']).to eq('Dmitriy Zaporozhets')
expect(contributor['commits']).to eq(13)
expect(contributor['email']).to eq('tiagonbotelho@hotmail.com')
expect(contributor['name']).to eq('tiagonbotelho')
expect(contributor['commits']).to eq(1)
expect(contributor['additions']).to eq(0)
expect(contributor['deletions']).to eq(0)
end
......
......@@ -3,53 +3,201 @@ require 'spec_helper'
describe API::Templates, api: true do
include ApiHelpers
context 'global templates' do
describe 'the Template Entity' do
before { get api('/gitignores/Ruby') }
shared_examples_for 'the Template Entity' do |path|
before { get api(path) }
it { expect(json_response['name']).to eq('Ruby') }
it { expect(json_response['content']).to include('*.gem') }
end
describe 'the TemplateList Entity' do
before { get api('/gitignores') }
shared_examples_for 'the TemplateList Entity' do |path|
before { get api(path) }
it { expect(json_response.first['name']).not_to be_nil }
it { expect(json_response.first['content']).to be_nil }
end
context 'requesting gitignores' do
describe 'GET /gitignores' do
shared_examples_for 'requesting gitignores' do |path|
it 'returns a list of available gitignore templates' do
get api('/gitignores')
get api(path)
expect(response.status).to eq(200)
expect(response).to have_http_status(200)
expect(json_response).to be_an Array
expect(json_response.size).to be > 15
end
end
end
context 'requesting gitlab-ci-ymls' do
describe 'GET /gitlab_ci_ymls' do
shared_examples_for 'requesting gitlab-ci-ymls' do |path|
it 'returns a list of available gitlab_ci_ymls' do
get api('/gitlab_ci_ymls')
get api(path)
expect(response.status).to eq(200)
expect(response).to have_http_status(200)
expect(json_response).to be_an Array
expect(json_response.first['name']).not_to be_nil
end
end
end
describe 'GET /gitlab_ci_ymls/Ruby' do
shared_examples_for 'requesting gitlab-ci-yml for Ruby' do |path|
it 'adds a disclaimer on the top' do
get api('/gitlab_ci_ymls/Ruby')
get api(path)
expect(response).to have_http_status(200)
expect(json_response['name']).not_to be_nil
expect(json_response['content']).to start_with("# This file is a template,")
end
end
shared_examples_for 'the License Template Entity' do |path|
before { get api(path) }
it 'returns a license template' do
expect(json_response['key']).to eq('mit')
expect(json_response['name']).to eq('MIT License')
expect(json_response['nickname']).to be_nil
expect(json_response['popular']).to be true
expect(json_response['html_url']).to eq('http://choosealicense.com/licenses/mit/')
expect(json_response['source_url']).to eq('https://opensource.org/licenses/MIT')
expect(json_response['description']).to include('A permissive license that is short and to the point.')
expect(json_response['conditions']).to eq(%w[include-copyright])
expect(json_response['permissions']).to eq(%w[commercial-use modifications distribution private-use])
expect(json_response['limitations']).to eq(%w[no-liability])
expect(json_response['content']).to include('The MIT License (MIT)')
end
end
shared_examples_for 'GET licenses' do |path|
it 'returns a list of available license templates' do
get api(path)
expect(response).to have_http_status(200)
expect(json_response).to be_an Array
expect(json_response.size).to eq(15)
expect(json_response.map { |l| l['key'] }).to include('agpl-3.0')
end
describe 'the popular parameter' do
context 'with popular=1' do
it 'returns a list of available popular license templates' do
get api("#{path}?popular=1")
expect(response).to have_http_status(200)
expect(json_response).to be_an Array
expect(json_response.size).to eq(3)
expect(json_response.map { |l| l['key'] }).to include('apache-2.0')
end
end
end
end
shared_examples_for 'GET licenses/:name' do |path|
context 'with :project and :fullname given' do
before do
get api("#{path}/#{license_type}?project=My+Awesome+Project&fullname=Anton+#{license_type.upcase}")
end
context 'for the mit license' do
let(:license_type) { 'mit' }
it 'returns the license text' do
expect(json_response['content']).to include('The MIT License (MIT)')
end
it 'replaces placeholder values' do
expect(json_response['content']).to include("Copyright (c) #{Time.now.year} Anton")
end
end
context 'for the agpl-3.0 license' do
let(:license_type) { 'agpl-3.0' }
it 'returns the license text' do
expect(json_response['content']).to include('GNU AFFERO GENERAL PUBLIC LICENSE')
end
it 'replaces placeholder values' do
expect(json_response['content']).to include('My Awesome Project')
expect(json_response['content']).to include("Copyright (C) #{Time.now.year} Anton")
end
end
context 'for the gpl-3.0 license' do
let(:license_type) { 'gpl-3.0' }
it 'returns the license text' do
expect(json_response['content']).to include('GNU GENERAL PUBLIC LICENSE')
end
it 'replaces placeholder values' do
expect(json_response['content']).to include('My Awesome Project')
expect(json_response['content']).to include("Copyright (C) #{Time.now.year} Anton")
end
end
context 'for the gpl-2.0 license' do
let(:license_type) { 'gpl-2.0' }
it 'returns the license text' do
expect(json_response['content']).to include('GNU GENERAL PUBLIC LICENSE')
end
it 'replaces placeholder values' do
expect(json_response['content']).to include('My Awesome Project')
expect(json_response['content']).to include("Copyright (C) #{Time.now.year} Anton")
end
end
context 'for the apache-2.0 license' do
let(:license_type) { 'apache-2.0' }
it 'returns the license text' do
expect(json_response['content']).to include('Apache License')
end
it 'replaces placeholder values' do
expect(json_response['content']).to include("Copyright #{Time.now.year} Anton")
end
end
context 'for an uknown license' do
let(:license_type) { 'muth-over9000' }
it 'returns a 404' do
expect(response).to have_http_status(404)
end
end
end
context 'with no :fullname given' do
context 'with an authenticated user' do
let(:user) { create(:user) }
it 'replaces the copyright owner placeholder with the name of the current user' do
get api('/templates/licenses/mit', user)
expect(json_response['content']).to include("Copyright (c) #{Time.now.year} #{user.name}")
end
end
end
end
describe 'with /templates namespace' do
it_behaves_like 'the Template Entity', '/templates/gitignores/Ruby'
it_behaves_like 'the TemplateList Entity', '/templates/gitignores'
it_behaves_like 'requesting gitignores', '/templates/gitignores'
it_behaves_like 'requesting gitlab-ci-ymls', '/templates/gitlab_ci_ymls'
it_behaves_like 'requesting gitlab-ci-yml for Ruby', '/templates/gitlab_ci_ymls/Ruby'
it_behaves_like 'the License Template Entity', '/templates/licenses/mit'
it_behaves_like 'GET licenses', '/templates/licenses'
it_behaves_like 'GET licenses/:name', '/templates/licenses'
end
describe 'without /templates namespace' do
it_behaves_like 'the Template Entity', '/gitignores/Ruby'
it_behaves_like 'the TemplateList Entity', '/gitignores'
it_behaves_like 'requesting gitignores', '/gitignores'
it_behaves_like 'requesting gitlab-ci-ymls', '/gitlab_ci_ymls'
it_behaves_like 'requesting gitlab-ci-yml for Ruby', '/gitlab_ci_ymls/Ruby'
it_behaves_like 'the License Template Entity', '/licenses/mit'
it_behaves_like 'GET licenses', '/licenses'
it_behaves_like 'GET licenses/:name', '/licenses'
end
end
......@@ -913,4 +913,58 @@ describe API::API, api: true do
expect(response).to have_http_status(404)
end
end
describe 'GET /user/:id/events' do
let(:user) { create(:user) }
let(:project) { create(:empty_project) }
let(:note) { create(:note_on_issue, note: 'What an awesome day!', project: project) }
before do
project.add_user(user, :developer)
EventCreateService.new.leave_note(note, user)
end
context "as a user than cannot see the event's project" do
it 'returns no events' do
other_user = create(:user)
get api("/users/#{user.id}/events", other_user)
expect(response).to have_http_status(200)
expect(json_response).to be_empty
end
end
context "as a user than can see the event's project" do
it_behaves_like 'a paginated resources' do
let(:request) { get api("/users/#{user.id}/events", user) }
end
context 'joined event' do
it 'returns the "joined" event' do
get api("/users/#{user.id}/events", user)
comment_event = json_response.find { |e| e['action_name'] == 'commented on' }
expect(comment_event['project_id'].to_i).to eq(project.id)
expect(comment_event['author_username']).to eq(user.username)
expect(comment_event['note']['id']).to eq(note.id)
expect(comment_event['note']['body']).to eq('What an awesome day!')
joined_event = json_response.find { |e| e['action_name'] == 'joined' }
expect(joined_event['project_id'].to_i).to eq(project.id)
expect(joined_event['author_username']).to eq(user.username)
expect(joined_event['author']['name']).to eq(user.name)
end
end
end
it 'returns a 404 error if not found' do
get api('/users/42/events', user)
expect(response).to have_http_status(404)
expect(json_response['message']).to eq('404 User Not Found')
end
end
end
......@@ -439,8 +439,8 @@ describe 'Git HTTP requests', lib: true do
before do
# Provide a dummy file in its place
allow_any_instance_of(Repository).to receive(:blob_at).and_call_original
allow_any_instance_of(Repository).to receive(:blob_at).with('5937ac0a7beb003549fc5fd26fc247adbce4a52e', 'info/refs') do
Gitlab::Git::Blob.find(project.repository, 'master', '.gitignore')
allow_any_instance_of(Repository).to receive(:blob_at).with('b83d6e391c22777fca1ed3012fce84f633d7fed0', 'info/refs') do
Gitlab::Git::Blob.find(project.repository, 'master', 'bar/branch-test.txt')
end
get "/#{project.path_with_namespace}/blob/master/info/refs"
......
......@@ -337,7 +337,7 @@ describe Projects::CommitsController, 'routing' do
end
it 'to #show' do
expect(get('/gitlab/gitlabhq/commits/master.atom')).to route_to('projects/commits#show', namespace_id: 'gitlab', project_id: 'gitlabhq', id: 'master', format: 'atom')
expect(get('/gitlab/gitlabhq/commits/master.atom')).to route_to('projects/commits#show', namespace_id: 'gitlab', project_id: 'gitlabhq', id: 'master.atom')
end
end
......
require 'spec_helper'
describe MergeRequests::AssignIssuesService, services: true do
let(:user) { create(:user) }
let(:project) { create(:project, :public) }
let(:issue) { create(:issue, project: project) }
let(:merge_request) { create(:merge_request, :simple, source_project: project, author: user, description: "fixes #{issue.to_reference}") }
let(:service) { described_class.new(project, user, merge_request: merge_request) }
before do
project.team << [user, :developer]
end
it 'finds unassigned issues fixed in merge request' do
expect(service.assignable_issues.map(&:id)).to include(issue.id)
end
it 'ignores issues already assigned to any user' do
issue.update!(assignee: create(:user))
expect(service.assignable_issues).to be_empty
end
it 'ignores issues the user cannot update assignee on' do
project.team.truncate
expect(service.assignable_issues).to be_empty
end
it 'ignores all issues unless current_user is merge_request.author' do
merge_request.update!(author: create(:user))
expect(service.assignable_issues).to be_empty
end
it 'accepts precomputed data for closes_issues' do
issue2 = create(:issue, project: project)
service2 = described_class.new(project,
user,
merge_request: merge_request,
closes_issues: [issue, issue2])
expect(service2.assignable_issues.count).to eq 2
end
it 'assigns these to the merge request owner' do
expect { service.execute }.to change { issue.reload.assignee }.to(user)
end
end
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment