Merge remote-tracking branch 'upstream/master' into enable-shared-runners-with-admins

* upstream/master: (120 commits)
  Update CHANGELOG for 8.9.4, 8.8.7, and 8.7.9.
  Remove additional entries from CHANGELOG
  Catch permission denied errors and ignore the disk
  Remove coveralls lines
  Make GH one-off auth the default again for importing GH projects
  Import from Github using Personal Access Tokens.
  Remove hardcoded gitlab-shell version in test env now that the required tag is published
  Updated breakpoint for sidebar pinning
  Expire branch/tag git data when needed.
  Remove unnecessary parens
  Enable Style/UnneededCapitalW Rubocop cop
  Expiry date on pinned nav cookie
  Fix broken spec in git_push_service_spec by stubbing an external issue tracker
  Handle external issues in IssueReferenceFilter
  Move Changelog entry for build retry fix to 8.9.4
  Add Changelog entry for build sidebar retry link fix
  Improve method that tells if build is retryable
  Do not show build retry link when build is active
  Remove coveralls as its unused
  Move changelot item "Add sub nav to file page view" to 8.9.4
  ...

.gitlab-ci.yml

@@ -18,6 +18,7 @@ variables:
   SIMPLECOV: "true"
   USE_DB: "true"
   USE_BUNDLE_INSTALL: "true"
+  GIT_DEPTH: "20"
 
 before_script:
   - source ./scripts/prepare_build.sh

.rubocop.yml

-require: rubocop-rspec
+require:
+  - rubocop-rspec
+  - ./rubocop/rubocop
 
 AllCops:
   TargetRubyVersion: 2.1
@@ -532,11 +534,11 @@ Style/SingleLineMethods:
 
 # Use spaces after colons.
 Style/SpaceAfterColon:
-  Enabled: false
+  Enabled: true
 
 # Use spaces after commas.
 Style/SpaceAfterComma:
-  Enabled: false
+  Enabled: true
 
 # Do not put a space between a method name and the opening parenthesis in a
 # method definition.
@@ -679,7 +681,7 @@ Style/UnlessElse:
 
 # Checks for %W when interpolation is not needed.
 Style/UnneededCapitalW:
-  Enabled: false
+  Enabled: true
 
 # TODO: Enable UnneededInterpolation Cop.
 # Checks for strings that are just an interpolated expression.

CHANGELOG

 Please view this file on the master branch, on stable branches it's out of date.
 
 v 8.10.0 (unreleased)
+  - Fix commit builds API, return all builds for all pipelines for given commit. !4849
   - Replace Haml with Hamlit to make view rendering faster. !3666
+  - Refactor repository paths handling to allow multiple git mount points
+  - Add Application Setting to configure default Repository Path for new projects
   - Wrap code blocks on Activies and Todos page. !4783 (winniehell)
+  - Align flash messages with left side of page content !4959 (winniehell)
+  - Display last commit of deleted branch in push events !4699 (winniehell)
+  - Apply the trusted_proxies config to the rack request object for use with rack_attack
   - Add Sidekiq queue duration to transaction metrics.
+  - Let Workhorse serve format-patch diffs
   - Make images fit to the size of the viewport !4810
   - Fix check for New Branch button on Issue page !4630 (winniehell)
   - Fix MR-auto-close text added to description. !4836
   - Fix pagination when sorting by columns with lots of ties (like priority)
   - Exclude email check from the standard health check
   - Fix changing issue state columns in milestone view
+  - Add notification settings dropdown for groups
+  - Allow importing from Github using Personal Access Tokens. (Eric K Idema)
   - Fix user creation with stronger minimum password requirements !4054 (nathan-pmt)
+  - PipelinesFinder uses git cache data
   - Check for conflicts with existing Project's wiki path when creating a new project.
+  - Remove unused front-end variable -> default_issues_tracker
+  - Better caching of git calls on ProjectsController#show.
   - Add API endpoint for a group issues !4520 (mahcsig)
+  - Add Bugzilla integration !4930 (iamtjg)
   - Allow [ci skip] to be in any case and allow [skip ci]. !4785 (simon_w)
+  - Add basic system information like memory and disk usage to the admin panel
+
+v 8.9.4
+  - Fix privilege escalation issue with OAuth external users.
+  - Ensure references to private repos aren't shown to logged-out users.
+  - Fixed search field blur not removing focus. !4704
+  - Resolve "Sub nav isn't showing on file view". !4890
+  - Fixes middle click and double request when navigating through the file browser. !4891
+  - Fixed URL on label button when filtering. !4897
+  - Fixed commit avatar alignment. !4933
+  - Do not show build retry link when build is active. !4967
+  - Fix restore Rake task warning message output. !4980
+  - Handle external issues in IssueReferenceFilter. !4988
+  - Expiry date on pinned nav cookie. !5009
+  - Updated breakpoint for sidebar pinning. !5019
+
+v 8.9.3
+  - Fix encrypted data backwards compatibility after upgrading attr_encrypted gem. !4963
+  - Fix rendering of commit notes. !4953
+  - Resolve "Pin should show up at 1280px min". !4947
+  - Switched mobile button icons to ellipsis and angle. !4944
+  - Correctly returns todo ID after creating todo. !4941
+  - Better debugging for memory killer middleware. !4936
+  - Remove duplicate new page btn from edit wiki. !4904
+  - Use clock_gettime for all performance timestamps. !4899
+  - Use memorized tags array when searching tags by name. !4859
+  - Fixed avatar alignment in new MR view. !4901
+  - Removed fade when filtering results. !4932
+  - Fix missing avatar on system notes. !4954
+  - Reduce overhead and optimize ProjectTeam#max_member_access performance. !4973
+  - Use update_columns to by_pass all the dirty code on active_record. !4985
+  - Fix restore Rake task warning message output !4980
 
 v 8.9.2
   - Fix visibility of snippets when searching.
@@ -64,6 +109,7 @@ v 8.9.1
   - Remove duplicate 'New Page' button on edit wiki page
 
 v 8.9.0
+  - Fix group visibility form layout in application settings
   - Fix builds API response not including commit data
   - Fix error when CI job variables key specified but not defined
   - Fix pipeline status when there are no builds in pipeline
@@ -159,6 +205,7 @@ v 8.9.0
   - Add Application Setting to configure Container Registry token expire delay (default 5min)
   - Cache assigned issue and merge request counts in sidebar nav
   - Use Knapsack only in CI environment
+  - Updated project creation page to match new UI #2542
   - Cache project build count in sidebar nav
   - Add milestone expire date to the right sidebar
   - Manually mark a issue or merge request as a todo
@@ -208,12 +255,21 @@ v 8.9.0
   - Filter parameters for request_uri value on instrumented transactions.
   - Remove duplicated keys add UNIQUE index to keys fingerprint column
   - ExtractsPath get ref_names from repository cache, if not there access git.
+  - Show a flash warning about the error detail of XHR requests which failed with status code 404 and 500
   - Cache user todo counts from TodoService
   - Ensure Todos counters doesn't count Todos for projects pending delete
   - Add left/right arrows horizontal navigation
   - Add tooltip to pin/unpin navbar
   - Add new sub nav style to Wiki and Graphs sub navigation
 
+v 8.8.7
+  - Fix privilege escalation issue with OAuth external users.
+  - Ensure references to private repos aren't shown to logged-out users.
+
+v 8.8.6
+  - Fix visibility of snippets when searching.
+  - Update omniauth-saml to 1.6.0 !4951
+
 v 8.8.5
   - Import GitHub repositories respecting the API rate limit !4166
   - Fix todos page throwing errors when you have a project pending deletion !4300
@@ -344,6 +400,14 @@ v 8.8.0
   - When creating a .gitignore file a dropdown with templates will be provided
   - Shows the issue/MR list search/filter form and corrects the mobile styling for guest users. #17562
 
+v 8.7.9
+  - Fix privilege escalation issue with OAuth external users.
+  - Ensure references to private repos aren't shown to logged-out users.
+
+v 8.7.8
+  - Fix visibility of snippets when searching.
+  - Update omniauth-saml to 1.6.0 !4951
+
 v 8.7.7
   - Fix import by `Any Git URL` broken if the URL contains a space
   - Prevent unauthorized access to other projects build traces

GITLAB_SHELL_VERSION

-3.0.0
+3.1.0

GITLAB_WORKHORSE_VERSION

-0.7.5
+0.7.7

Gemfile

@@ -91,6 +91,7 @@ gem 'fog-core', '~> 1.40'
 gem 'fog-local', '~> 0.3'
 gem 'fog-google', '~> 0.3'
 gem 'fog-openstack', '~> 0.1'
+gem 'fog-rackspace', '~> 0.1.1'
 
 # for aws storage
 gem "unf", '~> 0.1.4'
@@ -302,7 +303,6 @@ group :development, :test do
   gem 'rubocop', '~> 0.40.0', require: false
   gem 'rubocop-rspec', '~> 1.5.0', require: false
   gem 'scss_lint', '~> 0.47.0', require: false
-  gem 'coveralls', '~> 0.8.2', require: false
   gem 'simplecov', '~> 0.11.0', require: false
   gem 'flog', require: false
   gem 'flay', require: false
@@ -346,3 +346,7 @@ gem "paranoia", "~> 2.0"
 
 # Health check
 gem 'health_check', '~> 1.5.1'
+
+# System information
+gem 'vmstat', '~> 2.1.0'
+gem 'sys-filesystem', '~> 1.1.6'

Gemfile.lock

@@ -141,12 +141,6 @@ GEM
     colorize (0.7.7)
     concurrent-ruby (1.0.2)
     connection_pool (2.2.0)
-    coveralls (0.8.13)
-      json (~> 1.8)
-      simplecov (~> 0.11.0)
-      term-ansicolor (~> 1.3)
-      thor (~> 0.19.1)
-      tins (~> 1.6.0)
     crack (0.4.3)
       safe_yaml (~> 1.0.0)
     creole (0.5.0)
@@ -243,6 +237,11 @@ GEM
       fog-core (>= 1.39)
       fog-json (>= 1.0)
       ipaddress (>= 0.8)
+    fog-rackspace (0.1.1)
+      fog-core (>= 1.35)
+      fog-json (>= 1.0)
+      fog-xml (>= 0.1)
+      ipaddress (>= 0.8)
     fog-xml (0.1.2)
       fog-core
       nokogiri (~> 1.5, >= 1.5.11)
@@ -716,6 +715,8 @@ GEM
       activerecord (>= 4.1, < 5.1)
       state_machines-activemodel (>= 0.3.0)
     stringex (2.5.2)
+    sys-filesystem (1.1.6)
+      ffi
     systemu (2.6.5)
     task_list (1.0.2)
       html-pipeline
@@ -724,8 +725,6 @@ GEM
     teaspoon-jasmine (2.2.0)
       teaspoon (>= 1.0.0)
     temple (0.7.7)
-    term-ansicolor (1.3.2)
-      tins (~> 1.0)
     test_after_commit (0.4.2)
       activerecord (>= 3.2)
     thin (1.6.4)
@@ -746,7 +745,6 @@ GEM
       mime-types
       multi_json (~> 1.7)
       twitter-stream (~> 0.1)
-    tins (1.6.0)
     turbolinks (2.5.3)
       coffee-rails
     twitter-stream (0.1.16)
@@ -780,6 +778,7 @@ GEM
       coercible (~> 1.0)
       descendants_tracker (~> 0.0, >= 0.0.3)
       equalizer (~> 0.0, >= 0.0.9)
+    vmstat (2.1.0)
     warden (1.2.6)
       rack (>= 1.0)
     web-console (2.3.0)
@@ -835,7 +834,6 @@ DEPENDENCIES
   chronic_duration (~> 0.10.6)
   coffee-rails (~> 4.1.0)
   connection_pool (~> 2.0)
-  coveralls (~> 0.8.2)
   creole (~> 0.5.0)
   d3_rails (~> 3.5.0)
   database_cleaner (~> 1.4.0)
@@ -857,6 +855,7 @@ DEPENDENCIES
   fog-google (~> 0.3)
   fog-local (~> 0.3)
   fog-openstack (~> 0.1)
+  fog-rackspace (~> 0.1.1)
   font-awesome-rails (~> 4.6.1)
   foreman
   fuubar (~> 2.0.0)
@@ -969,6 +968,7 @@ DEPENDENCIES
   spring-commands-teaspoon (~> 0.0.2)
   sprockets (~> 3.6.0)
   state_machines-activerecord (~> 0.4.0)
+  sys-filesystem (~> 1.1.6)
   task_list (~> 1.0.2)
   teaspoon (~> 1.1.0)
   teaspoon-jasmine (~> 2.2.0)
@@ -984,6 +984,7 @@ DEPENDENCIES
   unicorn-worker-killer (~> 0.4.2)
   version_sorter (~> 2.0.0)
   virtus (~> 1.0.1)
+  vmstat (~> 2.1.0)
   web-console (~> 2.0)
   webmock (~> 1.21.0)
   wikicloth (= 0.8.1)

app/assets/javascripts/application.js.coffee

@@ -185,6 +185,15 @@ $ ->
       else
         buttons.enable()
 
+  $(document).ajaxError (e, xhrObj, xhrSetting, xhrErrorText) ->
+
+    if xhrObj.status is 401
+      new Flash 'You need to be logged in.', 'alert'
+
+    else if xhrObj.status in [ 404, 500 ]
+      new Flash 'Something went wrong on our end.', 'alert'
+
+
   # Show/Hide the profile menu when hovering the account box
   $('.account-box').hover -> $(@).toggleClass('hover')
 
@@ -199,7 +208,6 @@ $ ->
     $('.header-content .header-logo').toggle()
     $('.header-content .navbar-collapse').toggle()
     $('.navbar-toggle').toggleClass('active')
-    $('.navbar-toggle i').toggleClass("fa-angle-right fa-angle-left")
 
   # Show/hide comments on diff
   $body.on "click", ".js-toggle-diff-comments", (e) ->
@@ -261,8 +269,8 @@ $ ->
   new Aside()
 
   # Sidenav pinning
-  if $window.width() < 1440 and $.cookie('pin_nav') is 'true'
-    $.cookie('pin_nav', 'false', { path: '/' })
+  if $window.width() < 1024 and $.cookie('pin_nav') is 'true'
+    $.cookie('pin_nav', 'false', { path: '/', expires: 365 * 10 })
     $('.page-with-sidebar')
       .toggleClass('page-sidebar-collapsed page-sidebar-expanded')
       .removeClass('page-sidebar-pinned')
@@ -293,7 +301,7 @@ $ ->
                .toggleClass('header-collapsed header-expanded')
 
       # Save settings
-      $.cookie 'pin_nav', doPinNav, { path: '/' }
+      $.cookie 'pin_nav', doPinNav, { path: '/', expires: 365 * 10 }
 
       if $.cookie('pin_nav') is 'true' or doPinNav
         tooltipText = 'Unpin navigation'

app/assets/javascripts/dispatcher.js.coffee

@@ -84,6 +84,8 @@ class Dispatcher
         new Activities()
       when 'groups:show'
         shortcut_handler = new ShortcutsNavigation()
+        new NotificationsForm()
+        new NotificationsDropdown()
       when 'groups:group_members:index'
         new GroupMembers()
         new UsersSelect()

app/assets/javascripts/flash.js.coffee

@@ -4,11 +4,19 @@ class @Flash
     @flash.html("")
 
     innerDiv = $('<div/>',
-      class: "flash-#{type}",
-      text: message
+      class: "flash-#{type}"
     )
     innerDiv.appendTo(".flash-container")
 
+    textDiv = $("<div/>",
+      class: "flash-text",
+      text: message
+    )
+    textDiv.appendTo(innerDiv)
+
+    if @flash.parent().hasClass('content-wrapper')
+      textDiv.addClass('container-fluid container-limited')
+
     @flash.click -> $(@).fadeOut()
     @flash.show()
 

app/assets/javascripts/gl_dropdown.js.coffee

@@ -186,6 +186,8 @@ class GitLabDropdown
             @fullData = data
 
             @parseData @fullData
+
+            @filter.input.trigger('keyup') if @options.filterable and @filter and @filter.input
         }
 
     # Init filterable
@@ -218,6 +220,13 @@ class GitLabDropdown
     @dropdown.on 'keyup', (e) =>
       if e.which is 27 # Escape key
         $('.dropdown-menu-close', @dropdown).trigger 'click'
+    @dropdown.on 'blur', 'a', (e) =>
+      if e.relatedTarget?
+        $relatedTarget = $(e.relatedTarget)
+        $dropdownMenu = $relatedTarget.closest('.dropdown-menu')
+
+        if $dropdownMenu.length is 0
+          @dropdown.removeClass('open')
 
     if @dropdown.find(".dropdown-toggle-page").length
       @dropdown.find(".dropdown-toggle-page, .dropdown-menu-back").on "click", (e) =>

app/assets/javascripts/issuable.js.coffee

@@ -59,13 +59,12 @@ issuable_created = false
   filterResults: (form) =>
     formData = form.serialize()
 
-    $('.issues-holder, .merge-requests-holder').css('opacity', '0.5')
     formAction = form.attr('action')
     issuesUrl = formAction
     issuesUrl += ("#{if formAction.indexOf('?') < 0 then '?' else '&'}")
     issuesUrl += formData
 
-    Turbolinks.visit(issuesUrl);
+    Turbolinks.visit(issuesUrl)
 
   initChecks: ->
     @issuableBulkActions = $('.bulk-update').data('bulkActions')

app/assets/javascripts/lib/utils/text_utility.js.coffee

@@ -10,17 +10,41 @@
   gl.text.selectedText = (text, textarea) ->
     text.substring(textarea.selectionStart, textarea.selectionEnd)
 
-  gl.text.insertText = (textArea, text, tag, selected, wrap) ->
+  gl.text.lineBefore = (text, textarea) ->
+    split = text.substring(0, textarea.selectionStart).trim().split('\n')
+    split[split.length - 1]
+
+  gl.text.lineAfter = (text, textarea) ->
+    text.substring(textarea.selectionEnd).trim().split('\n')[0]
+
+  gl.text.blockTagText = (text, textArea, blockTag, selected) ->
+    lineBefore = @lineBefore(text, textArea)
+    lineAfter = @lineAfter(text, textArea)
+
+    if lineBefore is blockTag and lineAfter is blockTag
+      # To remove the block tag we have to select the line before & after
+      if blockTag?
+        textArea.selectionStart = textArea.selectionStart - (blockTag.length + 1)
+        textArea.selectionEnd = textArea.selectionEnd + (blockTag.length + 1)
+
+      selected
+    else
+      "#{blockTag}\n#{selected}\n#{blockTag}"
+
+  gl.text.insertText = (textArea, text, tag, blockTag, selected, wrap) ->
     selectedSplit = selected.split('\n')
     startChar = if not wrap and textArea.selectionStart > 0 then '\n' else ''
 
-    if selectedSplit.length > 1 and not wrap
-      insertText = selectedSplit.map((val) ->
-        if val.indexOf(tag) is 0
-          "#{val.replace(tag, '')}"
-        else
-          "#{tag}#{val}"
-      ).join('\n')
+    if selectedSplit.length > 1 and (not wrap or blockTag?)
+      if blockTag?
+        insertText = @blockTagText(text, textArea, blockTag, selected)
+      else
+        insertText = selectedSplit.map((val) ->
+          if val.indexOf(tag) is 0
+            "#{val.replace(tag, '')}"
+          else
+            "#{tag}#{val}"
+        ).join('\n')
     else
       insertText = "#{startChar}#{tag}#{selected}#{if wrap then tag else ' '}"
 
@@ -51,7 +75,7 @@
 
       textArea.setSelectionRange pos, pos
 
-  gl.text.updateText = (textArea, tag, wrap) ->
+  gl.text.updateText = (textArea, tag, blockTag, wrap) ->
     $textArea = $(textArea)
     oldVal = $textArea.val()
     textArea = $textArea.get(0)
@@ -59,7 +83,7 @@
     selected = @selectedText(text, textArea)
     $textArea.focus()
 
-    @insertText(textArea, text, tag, selected, wrap)
+    @insertText(textArea, text, tag, blockTag, selected, wrap)
 
   gl.text.init = (form) ->
     self = @
@@ -70,6 +94,7 @@
         self.updateText(
           $this.closest('.md-area').find('textarea'),
           $this.data('md-tag'),
+          $this.data('md-block'),
           not $this.data('md-prepend')
         )
 

app/assets/javascripts/search_autocomplete.js.coffee

@@ -171,22 +171,15 @@ class @SearchAutocomplete
     }
 
   bindEvents: ->
-    $(document).on 'click', @onDocumentClick
     @searchInput.on 'keydown', @onSearchInputKeyDown
     @searchInput.on 'keyup', @onSearchInputKeyUp
     @searchInput.on 'click', @onSearchInputClick
     @searchInput.on 'focus', @onSearchInputFocus
+    @searchInput.on 'blur', @onSearchInputBlur
     @clearInput.on 'click', @onClearInputClick
     @locationBadgeEl.on 'click', =>
       @searchInput.focus()
 
-  onDocumentClick: (e) =>
-    # If clicking outside the search box
-    # And search input is not focused
-    # And we are not clicking inside a suggestion
-    if not $.contains(@dropdown[0], e.target) and @isFocused and not $(e.target).closest('.search-form').length
-      @onSearchInputBlur()
-
   enableAutocomplete: ->
     # No need to enable anything if user is not logged in
     return if !gon.current_user_id
@@ -287,8 +280,6 @@ class @SearchAutocomplete
         value: @originalState._location
       )
 
-    @dropdown.removeClass 'open'
-
   badgePresent: ->
     @locationBadgeEl.length
 

app/assets/javascripts/shortcuts.js.coffee

@@ -9,12 +9,12 @@ class @Shortcuts
 
   onToggleHelp: (e) =>
     e.preventDefault()
-    @toggleHelp(@enabledHelp)
+    Shortcuts.toggleHelp(@enabledHelp)
 
-  toggleMarkdownPreview: (e) =>
+  toggleMarkdownPreview: (e) ->
     $(document).triggerHandler('markdown-preview:toggle', [e])
 
-  toggleHelp: (location) ->
+  @toggleHelp: (location) ->
     $modal = $('#modal-shortcuts')
 
     if $modal.length

app/assets/javascripts/tree.js.coffee

@@ -5,9 +5,15 @@ class @TreeView
     # Code browser tree slider
     # Make the entire tree-item row clickable, but not if clicking another link (like a commit message)
     $(".tree-content-holder .tree-item").on 'click', (e) ->
-      if (e.target.nodeName != "A")
-        path = $('.tree-item-file-name a', this).attr('href')
-        Turbolinks.visit(path)
+      $clickedEl = $(e.target)
+      path = $('.tree-item-file-name a', this).attr('href')
+
+      if not $clickedEl.is('a') and not $clickedEl.is('.str-truncated')
+        if e.metaKey or e.which is 2
+          e.preventDefault()
+          window.open path, '_blank'
+        else
+          Turbolinks.visit path
 
     # Show the "Loading commit data" for only the first element
     $('span.log_loading:first').removeClass('hide')

app/assets/stylesheets/framework/blocks.scss

@@ -137,7 +137,7 @@
     margin: 0;
     font-size: 24px;
     font-weight: normal;
-    margin-bottom: 5px;
+    margin-bottom: 10px;
     color: #4c4e54;
     font-size: 23px;
     line-height: 1.1;

app/assets/stylesheets/framework/flash.scss

@@ -16,4 +16,11 @@
     @extend .alert-danger;
     margin: 0;
   }
+
+  .flash-notice, .flash-alert {
+    .container-fluid.flash-text {
+      background: transparent;
+    }
+  }
 }
+

app/assets/stylesheets/framework/markdown_area.scss

@@ -125,7 +125,8 @@
   border: 0;
   outline: 0;
 
-  &:hover {
+  &:hover,
+  &:focus {
     color: $gl-link-color;
   }
 }

app/assets/stylesheets/framework/nav.scss

@@ -21,9 +21,8 @@
 
   .fa {
     position: relative;
-    top: 3px;
-    font-size: 13px;
-    color: $btn-placeholder-gray;
+    top: 5px;
+    font-size: 18px;
   }
 }
 

app/assets/stylesheets/framework/sidebar.scss

 .page-with-sidebar {
   padding-top: $header-height;
+  padding-bottom: 25px;
   transition: padding $sidebar-transition-duration;
 
   .sidebar-wrapper {

app/assets/stylesheets/framework/variables.scss

@@ -7,7 +7,7 @@ $gutter_collapsed_width: 62px;
 $gutter_width: 290px;
 $gutter_inner_width: 258px;
 $sidebar-transition-duration: .15s;
-$sidebar-breakpoint: 1440px;
+$sidebar-breakpoint: 1024px;
 
 /*
  * UI elements

app/assets/stylesheets/pages/commits.scss

@@ -83,11 +83,7 @@
   position: relative;
 
   @media (min-width: $screen-sm-min) {
-    padding-left: 20px;
-
-    .commit-info-block {
-      padding-left: 44px;
-    }
+    padding-left: 46px;
   }
 
   &:not(:last-child) {
@@ -102,9 +98,7 @@
 
 
   .avatar {
-    position: absolute;
-    top: 10px;
-    left: 16px;
+    margin-left: -46px;
   }
 
   .item-title {

app/assets/stylesheets/pages/groups.scss

@@ -48,11 +48,7 @@
 
   .access-request-button {
     @include btn-gray;
-    position: absolute;
-    right: 16px;
-    bottom: 32px;
-    padding: 3px 10px;
+    margin-right: 10px;
     text-transform: none;
-    background-color: $background-color;
   }
 }

app/assets/stylesheets/pages/merge_requests.scss

@@ -264,8 +264,15 @@
       margin-bottom: 4px;
     }
 
+    .item-title {
+      @media (min-width: $screen-sm-min) {
+        width: 49%;
+      }
+    }
+
     .avatar {
-      margin-left: 0;
+      left: 0;
+      top: 2px;
     }
 
     .commit-row-info {

app/assets/stylesheets/pages/notes.scss

@@ -41,6 +41,10 @@ ul.notes {
     .timeline-icon {
       .avatar {
         visibility: hidden;
+
+        .discussion-body & {
+          visibility: visible;
+        }
       }
     }
   }

app/assets/stylesheets/pages/projects.scss

@@ -13,10 +13,53 @@
 
 .new_project,
 .edit-project {
-  fieldset.features {
-    .control-label {
+  fieldset {
+    &.features .control-label {
       font-weight: normal;
     }
+    .form-group {
+      margin-bottom: 5px;
+    }
+    &> .form-group {
+      padding-left: 0;
+    }
+  }
+  .help-block {
+    margin-bottom: 10px;
+  }
+  .project-path {
+    padding-right: 0;
+    .form-control {
+      border-radius: $border-radius-base;
+    }
+  }
+  .input-group > div {
+    &:last-child {
+      padding-right: 0;
+    }
+  }
+  @media (max-width: $screen-xs-max) {
+    .input-group > div {
+      margin-bottom: 14px;
+      &:last-child {
+        margin-bottom: 0;
+      }
+    }
+    fieldset > .form-group:first-child {
+      padding-right: 0;
+    }
+  }
+
+  .input-group-addon {
+    &.static-namespace {
+      height: 35px;
+      border-radius: 3px;
+      border: 1px solid #e5e5e5;
+    }
+    &+ .select2 a {
+      border-top-left-radius: 0;
+      border-bottom-left-radius: 0;
+    }
   }
 }
 
@@ -365,10 +408,28 @@ a.deploy-project-label {
   }
 }
 
-.project-import .btn {
-  float: left;
-  margin-bottom: 10px;
-  margin-right: 10px;
+.project-import {
+  .form-group {
+    margin-bottom: 0;
+  }
+  .import-buttons {
+    padding-left: 0;
+    display: -webkit-flex;
+    display: flex;
+    -webkit-flex-wrap: wrap;
+    flex-wrap: wrap;
+    .btn {
+      margin-right: 10px;
+      padding: 8px 12px;
+    }
+    &> div {
+      margin-bottom: 14px;
+      padding-left: 0;
+      &:last-child {
+        margin-bottom: 0;
+      }
+    }
+  }
 }
 
 .project-stats {

app/assets/stylesheets/pages/tree.scss

@@ -101,7 +101,8 @@
   margin: 0;
 
   .commit {
-    padding: 0 0 0 55px;
+    padding-top: 0;
+    padding-bottom: 0;
 
     .commit-row-title {
       .commit-row-message {

app/controllers/admin/application_settings_controller.rb

@@ -109,6 +109,7 @@ class Admin::ApplicationSettingsController < Admin::ApplicationController
       :metrics_packet_size,
       :send_user_confirmation_email,
       :container_registry_token_expire_delay,
+      :repository_storage,
       restricted_visibility_levels: [],
       import_sources: [],
       disabled_oauth_sign_in_sources: []

app/controllers/admin/system_info_controller.rb

+class Admin::SystemInfoController < Admin::ApplicationController
+  EXCLUDED_MOUNT_OPTIONS = [
+    'nobrowse',
+    'read-only',
+    'ro'
+  ]
+
+  EXCLUDED_MOUNT_TYPES = [
+    'autofs',
+    'binfmt_misc',
+    'cgroup',
+    'debugfs',
+    'devfs',
+    'devpts',
+    'devtmpfs',
+    'efivarfs',
+    'fuse.gvfsd-fuse',
+    'fuseblk',
+    'fusectl',
+    'hugetlbfs',
+    'mqueue',
+    'proc',
+    'pstore',
+    'securityfs',
+    'sysfs',
+    'tmpfs',
+    'tracefs',
+    'vfat'
+  ]
+
+  def show
+    system_info = Vmstat.snapshot
+    mounts = Sys::Filesystem.mounts
+
+    @disks = []
+    mounts.each do |mount|
+      mount_options = mount.options.split(',')
+
+      next if (EXCLUDED_MOUNT_OPTIONS & mount_options).any?
+      next if (EXCLUDED_MOUNT_TYPES & [mount.mount_type]).any?
+
+      begin
+        disk = Sys::Filesystem.stat(mount.mount_point)
+        @disks.push({
+          bytes_total: disk.bytes_total,
+          bytes_used:  disk.bytes_used,
+          disk_name:   mount.name,
+          mount_path:  disk.path
+        })
+      rescue Sys::Filesystem::Error
+      end
+    end
+
+    @cpus = system_info.cpus.length
+
+    @mem_used = system_info.memory.active_bytes
+    @mem_total = system_info.memory.total_bytes
+  end
+end

app/controllers/ci/projects_controller.rb

@@ -25,7 +25,7 @@ module Ci
       return render_404 unless @project
 
       image = Ci::ImageForBuildService.new.execute(@project, params)
-      send_file image.path, filename: image.name, disposition: 'inline', type:"image/svg+xml"
+      send_file image.path, filename: image.name, disposition: 'inline', type: "image/svg+xml"
     end
 
     protected

app/controllers/groups_controller.rb

@@ -37,15 +37,12 @@ class GroupsController < Groups::ApplicationController
   end
 
   def show
-    @last_push = current_user.recent_push if current_user
-
-    @projects = @projects.includes(:namespace)
-    @projects = @projects.sorted_by_activity
-    @projects = filter_projects(@projects)
-    @projects = @projects.sort(@sort = params[:sort])
-    @projects = @projects.page(params[:page]) if params[:filter_projects].blank?
+    if current_user
+      @last_push            = current_user.recent_push
+      @notification_setting = current_user.notification_settings_for(group)
+    end
 
-    @shared_projects = GroupProjectsFinder.new(group, only_shared: true).execute(current_user)
+    setup_projects
 
     respond_to do |format|
       format.html
@@ -97,6 +94,16 @@ class GroupsController < Groups::ApplicationController
 
   protected
 
+  def setup_projects
+    @projects = @projects.includes(:namespace)
+    @projects = @projects.sorted_by_activity
+    @projects = filter_projects(@projects)
+    @projects = @projects.sort(@sort = params[:sort])
+    @projects = @projects.page(params[:page]) if params[:filter_projects].blank?
+
+    @shared_projects = GroupProjectsFinder.new(group, only_shared: true).execute(current_user)
+  end
+
   def authorize_create_group!
     unless can?(current_user, :create_group, nil)
       return render_404

app/controllers/import/github_controller.rb

 class Import::GithubController < Import::BaseController
   before_action :verify_github_import_enabled
-  before_action :github_auth, except: :callback
+  before_action :github_auth, only: [:status, :jobs, :create]
 
   rescue_from Octokit::Unauthorized, with: :github_unauthorized
 
+  helper_method :logged_in_with_github?
+
+  def new
+    if logged_in_with_github?
+      go_to_github_for_permissions
+    elsif session[:github_access_token]
+      redirect_to status_import_github_url
+    end
+  end
+
   def callback
     session[:github_access_token] = client.get_token(params[:code])
     redirect_to status_import_github_url
   end
 
+  def personal_access_token
+    session[:github_access_token] = params[:personal_access_token]
+    redirect_to status_import_github_url
+  end
+
   def status
     @repos = client.repos
     @already_added_projects = current_user.created_projects.where(import_type: "github")
@@ -57,10 +72,14 @@ class Import::GithubController < Import::BaseController
   end
 
   def github_unauthorized
-    go_to_github_for_permissions
+    session[:github_access_token] = nil
+    redirect_to new_import_github_url,
+      alert: 'Access denied to your GitHub account.'
   end
 
-  private
+  def logged_in_with_github?
+    current_user.identities.exists?(provider: 'github')
+  end
 
   def access_params
     { github_access_token: session[:github_access_token] }

app/controllers/notification_settings_controller.rb

@@ -2,11 +2,9 @@ class NotificationSettingsController < ApplicationController
   before_action :authenticate_user!
 
   def create
-    project = Project.find(params[:project][:id])
+    return render_404 unless can_read?(resource)
 
-    return render_404 unless can?(current_user, :read_project, project)
-
-    @notification_setting = current_user.notification_settings_for(project)
+    @notification_setting = current_user.notification_settings_for(resource)
     @saved = @notification_setting.update_attributes(notification_setting_params)
 
     render_response
@@ -21,6 +19,22 @@ class NotificationSettingsController < ApplicationController
 
   private
 
+  def resource
+    @resource ||=
+      if params[:project_id].present?
+        Project.find(params[:project_id])
+      elsif params[:namespace_id].present?
+        Group.find(params[:namespace_id])
+      end
+  end
+
+  def can_read?(resource)
+    ability_name = resource.class.name.downcase
+    ability_name = "read_#{ability_name}".to_sym
+
+    can?(current_user, ability_name, resource)
+  end
+
   def render_response
     render json: {
       html: view_to_html_string("shared/notifications/_button", notification_setting: @notification_setting),

app/controllers/projects/merge_requests_controller.rb

@@ -59,7 +59,13 @@ class Projects::MergeRequestsController < Projects::ApplicationController
     respond_to do |format|
       format.html
       format.json   { render json: @merge_request }
-      format.patch  { render text: @merge_request.to_patch }
+      format.patch  do
+        headers.store(*Gitlab::Workhorse.send_git_patch(@project.repository,
+                                                        @merge_request.diff_base_commit.id,
+                                                        @merge_request.last_commit.id))
+        headers['Content-Disposition'] = 'inline'
+        head :ok
+      end
       format.diff do
         return render_404 unless @merge_request.diff_refs
 

app/finders/pipelines_finder.rb

@@ -29,10 +29,10 @@ class PipelinesFinder
   end
 
   def branches
-    project.repository.branches.map(&:name)
+    project.repository.branch_names
   end
 
   def tags
-    project.repository.tags.map(&:name)
+    project.repository.tag_names
   end
 end

app/helpers/application_settings_helper.rb

@@ -78,4 +78,12 @@ module ApplicationSettingsHelper
       end
     end
   end
+
+  def repository_storage_options_for_select
+    options = Gitlab.config.repositories.storages.map do |name, path|
+      ["#{name} - #{path}", name]
+    end
+
+    options_for_select(options, @application_setting.repository_storage)
+  end
 end

app/helpers/blob_helper.rb

 module BlobHelper
-  def highlighter(blob_name, blob_content, nowrap: false)
-    Gitlab::Highlight.new(blob_name, blob_content, nowrap: nowrap)
+  def highlighter(blob_name, blob_content, repository: nil, nowrap: false)
+    Gitlab::Highlight.new(blob_name, blob_content, nowrap: nowrap, repository: repository)
   end
 
-  def highlight(blob_name, blob_content, nowrap: false, plain: false)
-    Gitlab::Highlight.highlight(blob_name, blob_content, nowrap: nowrap, plain: plain)
+  def highlight(blob_name, blob_content, repository: nil, nowrap: false, plain: false)
+    Gitlab::Highlight.highlight(blob_name, blob_content, nowrap: nowrap, plain: plain, repository: repository)
   end
 
   def no_highlight_files

app/helpers/dropdowns_helper.rb

@@ -69,7 +69,7 @@ module DropdownsHelper
 
   def dropdown_filter(placeholder, search_id: nil)
     content_tag :div, class: "dropdown-input" do
-      filter_output = search_field_tag search_id, nil, class: "dropdown-input-field", placeholder: placeholder
+      filter_output = search_field_tag search_id, nil, class: "dropdown-input-field", placeholder: placeholder, autocomplete: 'off'
       filter_output << icon('search', class: "dropdown-input-search")
       filter_output << icon('times', class: "dropdown-input-clear js-dropdown-input-clear", role: "button")
 

app/helpers/labels_helper.rb

@@ -34,10 +34,7 @@ module LabelsHelper
   # Returns a String
   def link_to_label(label, project: nil, type: :issue, tooltip: true, css_class: nil, &block)
     project ||= @project || label.project
-    link = send("namespace_project_#{type.to_s.pluralize}_path",
-                project.namespace,
-                project,
-                label_name: [label.name])
+    link = label_filter_path(project, label, type: type)
 
     if block_given?
       link_to link, class: css_class, &block
@@ -46,6 +43,13 @@ module LabelsHelper
     end
   end
 
+  def label_filter_path(project, label, type: issue)
+    send("namespace_project_#{type.to_s.pluralize}_path",
+                project.namespace,
+                project,
+                label_name: [label.name])
+  end
+
   def project_label_names
     @project.labels.pluck(:title)
   end

app/helpers/notes_helper.rb

@@ -69,4 +69,14 @@ module NotesHelper
     button_tag 'Reply...', class: 'btn btn-text-field js-discussion-reply-button',
                            data: data, title: 'Add a reply'
   end
+
+  def note_max_access_for_user(note)
+    @max_access_by_user_id ||= Hash.new do |hash, key|
+      project = key[:project]
+      hash[key] = project.team.human_max_access(key[:user_id])
+    end
+
+    full_key = { project: note.project, user_id: note.author_id }
+    @max_access_by_user_id[full_key]
+  end
 end

app/helpers/notifications_helper.rb

@@ -72,6 +72,6 @@ module NotificationsHelper
   # Create hidden field to send notification setting source to controller
   def hidden_setting_source_input(notification_setting)
     return unless notification_setting.source_type
-    hidden_field_tag "#{notification_setting.source_type.downcase}[id]", notification_setting.source_id
+    hidden_field_tag "#{notification_setting.source_type.downcase}_id", notification_setting.source_id
   end
 end

app/helpers/page_layout_helper.rb

@@ -52,7 +52,7 @@ module PageLayoutHelper
     raise ArgumentError, 'cannot provide more than two attributes' if map.length > 2
 
     @page_card_attributes ||= {}
-    @page_card_attributes = map.reject { |_,v| v.blank? } if map.present?
+    @page_card_attributes = map.reject { |_, v| v.blank? } if map.present?
     @page_card_attributes
   end
 

app/helpers/projects_helper.rb

@@ -15,7 +15,7 @@ module ProjectsHelper
   def link_to_member_avatar(author, opts = {})
     default_opts = { avatar: true, name: true, size: 16, author_class: 'author', title: ":name" }
     opts = default_opts.merge(opts)
-    image_tag(avatar_icon(author, opts[:size]), width: opts[:size], class: "avatar avatar-inline #{"s#{opts[:size]}" if opts[:size]}", alt:'') if opts[:avatar]
+    image_tag(avatar_icon(author, opts[:size]), width: opts[:size], class: "avatar avatar-inline #{"s#{opts[:size]}" if opts[:size]}", alt: '') if opts[:avatar]
   end
 
   def link_to_member(project, author, opts = {}, &block)
@@ -27,7 +27,7 @@ module ProjectsHelper
     author_html =  ""
 
     # Build avatar image tag
-    author_html << image_tag(avatar_icon(author, opts[:size]), width: opts[:size], class: "avatar avatar-inline #{"s#{opts[:size]}" if opts[:size]}", alt:'') if opts[:avatar]
+    author_html << image_tag(avatar_icon(author, opts[:size]), width: opts[:size], class: "avatar avatar-inline #{"s#{opts[:size]}" if opts[:size]}", alt: '') if opts[:avatar]
 
     # Build name span tag
     if opts[:by_username]
@@ -327,9 +327,9 @@ module ProjectsHelper
     end
   end
 
-  def sanitize_repo_path(message)
+  def sanitize_repo_path(project, message)
     return '' unless message.present?
 
-    message.strip.gsub(Gitlab.config.gitlab_shell.repos_path.chomp('/'), "[REPOS PATH]")
+    message.strip.gsub(project.repository_storage_path.chomp('/'), "[REPOS PATH]")
   end
 end

app/models/application_setting.rb

@@ -55,6 +55,10 @@ class ApplicationSetting < ActiveRecord::Base
             presence: true,
             numericality: { only_integer: true, greater_than: 0 }
 
+  validates :repository_storage,
+    presence: true,
+    inclusion: { in: ->(_object) { Gitlab.config.repositories.storages.keys } }
+
   validates_each :restricted_visibility_levels do |record, attr, value|
     unless value.nil?
       value.each do |level|
@@ -134,6 +138,7 @@ class ApplicationSetting < ActiveRecord::Base
       disabled_oauth_sign_in_sources: [],
       send_user_confirmation_email: false,
       container_registry_token_expire_delay: 5,
+      repository_storage: 'default',
     )
   end
 

app/models/ci/build.rb

@@ -90,7 +90,7 @@ module Ci
     end
 
     def retryable?
-      project.builds_enabled? && commands.present?
+      project.builds_enabled? && commands.present? && complete?
     end
 
     def retried?

app/models/ci/variable.rb

@@ -13,6 +13,7 @@ module Ci
 
     attr_encrypted :value, 
        mode: :per_attribute_iv_and_salt,
+       insecure_mode: true,
        key: Gitlab::Application.secrets.db_key_base,
        algorithm: 'aes-256-cbc'
   end

app/models/event.rb

@@ -315,7 +315,7 @@ class Event < ActiveRecord::Base
 
   def body?
     if push?
-      push_with_commits?
+      push_with_commits? || rm_ref?
     elsif note?
       true
     else

app/models/member.rb

@@ -32,6 +32,7 @@ class Member < ActiveRecord::Base
   scope :request, -> { where.not(requested_at: nil) }
   scope :non_request, -> { where(requested_at: nil) }
   scope :non_pending, -> { non_request.non_invite }
+  scope :has_access, -> { where('access_level > 0') }
 
   scope :guests, -> { where(access_level: GUEST) }
   scope :reporters, -> { where(access_level: REPORTER) }

app/models/merge_request.rb

@@ -319,13 +319,6 @@ class MergeRequest < ActiveRecord::Base
     )
   end
 
-  # Returns the commit as a series of email patches.
-  #
-  # see "git format-patch"
-  def to_patch
-    target_project.repository.format_patch(diff_base_commit.sha, source_sha)
-  end
-
   def hook_attrs
     attrs = {
       source: source_project.try(:hook_attrs),

app/models/merge_request_diff.rb

@@ -108,44 +108,46 @@ class MergeRequestDiff < ActiveRecord::Base
   # Reload all commits related to current merge request from repo
   # and save it as array of hashes in st_commits db field
   def reload_commits
+    new_attributes = {}
+
     commit_objects = unmerged_commits
 
     if commit_objects.present?
-      self.st_commits = dump_commits(commit_objects)
+      new_attributes[:st_commits] = dump_commits(commit_objects)
     end
 
-    save
+    update_columns_serialized(new_attributes)
   end
 
   # Reload diffs between branches related to current merge request from repo
   # and save it as array of hashes in st_diffs db field
   def reload_diffs
+    new_attributes = {}
     new_diffs = []
 
     if commits.size.zero?
-      self.state = :empty
+      new_attributes[:state] = :empty
     else
       diff_collection = unmerged_diffs
 
       if diff_collection.overflow?
         # Set our state to 'overflow' to make the #empty? and #collected?
         # methods (generated by StateMachine) return false.
-        self.state = :overflow
+        new_attributes[:state] = :overflow
       end
 
-      self.real_size = diff_collection.real_size
+      new_attributes[:real_size] = diff_collection.real_size
 
       if diff_collection.any?
         new_diffs = dump_diffs(diff_collection)
-        self.state = :collected
+        new_attributes[:state] = :collected
       end
     end
 
-    self.st_diffs = new_diffs
-
-    self.base_commit_sha = self.repository.merge_base(self.head, self.base)
+    new_attributes[:st_diffs] = new_diffs
+    new_attributes[:base_commit_sha] = self.repository.merge_base(self.head, self.base)
 
-    self.save
+    update_columns_serialized(new_attributes)
   end
 
   # Collect array of Git::Diff objects
@@ -190,4 +192,29 @@ class MergeRequestDiff < ActiveRecord::Base
         )
       end
   end
+
+  private
+
+  #
+  # #save or #update_attributes providing changes on serialized attributes do a lot of
+  # serialization and deserialization calls resulting in bad performance.
+  # Using #update_columns solves the problem with just one YAML.dump per serialized attribute that we provide.
+  # As a tradeoff we need to reload the current instance to properly manage time objects on those serialized
+  # attributes. So to keep the same behaviour as the attribute assignment we reload the instance.
+  # The difference is in the usage of
+  # #write_attribute= (#update_attributes) and #raw_write_attribute= (#update_columns)
+  #
+  # Ex:
+  #
+  #   new_attributes[:st_commits].first.slice(:committed_date)
+  #   => {:committed_date=>2014-02-27 11:01:38 +0200}
+  #   YAML.load(YAML.dump(new_attributes[:st_commits].first.slice(:committed_date)))
+  #   => {:committed_date=>2014-02-27 10:01:38 +0100}
+  #
+  def update_columns_serialized(new_attributes)
+    return unless new_attributes.any?
+
+    update_columns(new_attributes.merge(updated_at: current_time_from_proper_timezone))
+    reload
+  end
 end

app/models/namespace.rb

@@ -21,8 +21,10 @@ class Namespace < ActiveRecord::Base
 
   delegate :name, to: :owner, allow_nil: true, prefix: true
 
-  after_create :ensure_dir_exist
   after_update :move_dir, if: :path_changed?
+
+  # Save the storage paths before the projects are destroyed to use them on after destroy
+  before_destroy(prepend: true) { @old_repository_storage_paths = repository_storage_paths }
   after_destroy :rm_dir
 
   scope :root, -> { where('type IS NULL') }
@@ -87,51 +89,35 @@ class Namespace < ActiveRecord::Base
     owner_name
   end
 
-  def ensure_dir_exist
-    gitlab_shell.add_namespace(path)
-  end
-
-  def rm_dir
-    # Move namespace directory into trash.
-    # We will remove it later async
-    new_path = "#{path}+#{id}+deleted"
-
-    if gitlab_shell.mv_namespace(path, new_path)
-      message = "Namespace directory \"#{path}\" moved to \"#{new_path}\""
-      Gitlab::AppLogger.info message
-
-      # Remove namespace directroy async with delay so
-      # GitLab has time to remove all projects first
-      GitlabShellWorker.perform_in(5.minutes, :rm_namespace, new_path)
-    end
-  end
-
   def move_dir
-    # Ensure old directory exists before moving it
-    gitlab_shell.add_namespace(path_was)
-
     if any_project_has_container_registry_tags?
       raise Exception.new('Namespace cannot be moved, because at least one project has tags in container registry')
     end
 
-    if gitlab_shell.mv_namespace(path_was, path)
-      Gitlab::UploadsTransfer.new.rename_namespace(path_was, path)
-
-      # If repositories moved successfully we need to
-      # send update instructions to users.
-      # However we cannot allow rollback since we moved namespace dir
-      # So we basically we mute exceptions in next actions
-      begin
-        send_update_instructions
-      rescue
-        # Returning false does not rollback after_* transaction but gives
-        # us information about failing some of tasks
-        false
+    # Move the namespace directory in all storages paths used by member projects
+    repository_storage_paths.each do |repository_storage_path|
+      # Ensure old directory exists before moving it
+      gitlab_shell.add_namespace(repository_storage_path, path_was)
+
+      unless gitlab_shell.mv_namespace(repository_storage_path, path_was, path)
+        # if we cannot move namespace directory we should rollback
+        # db changes in order to prevent out of sync between db and fs
+        raise Exception.new('namespace directory cannot be moved')
       end
-    else
-      # if we cannot move namespace directory we should rollback
-      # db changes in order to prevent out of sync between db and fs
-      raise Exception.new('namespace directory cannot be moved')
+    end
+
+    Gitlab::UploadsTransfer.new.rename_namespace(path_was, path)
+
+    # If repositories moved successfully we need to
+    # send update instructions to users.
+    # However we cannot allow rollback since we moved namespace dir
+    # So we basically we mute exceptions in next actions
+    begin
+      send_update_instructions
+    rescue
+      # Returning false does not rollback after_* transaction but gives
+      # us information about failing some of tasks
+      false
     end
   end
 
@@ -152,4 +138,33 @@ class Namespace < ActiveRecord::Base
   def find_fork_of(project)
     projects.joins(:forked_project_link).find_by('forked_project_links.forked_from_project_id = ?', project.id)
   end
+
+  private
+
+  def repository_storage_paths
+    # We need to get the storage paths for all the projects, even the ones that are
+    # pending delete. Unscoping also get rids of the default order, which causes
+    # problems with SELECT DISTINCT.
+    Project.unscoped do
+      projects.select('distinct(repository_storage)').to_a.map(&:repository_storage_path)
+    end
+  end
+
+  def rm_dir
+    # Remove the namespace directory in all storages paths used by member projects
+    @old_repository_storage_paths.each do |repository_storage_path|
+      # Move namespace directory into trash.
+      # We will remove it later async
+      new_path = "#{path}+#{id}+deleted"
+
+      if gitlab_shell.mv_namespace(repository_storage_path, path, new_path)
+        message = "Namespace directory \"#{path}\" moved to \"#{new_path}\""
+        Gitlab::AppLogger.info message
+
+        # Remove namespace directroy async with delay so
+        # GitLab has time to remove all projects first
+        GitlabShellWorker.perform_in(5.minutes, :rm_namespace, repository_storage_path, new_path)
+      end
+    end
+  end
 end

app/models/network/graph.rb

@@ -54,7 +54,7 @@ module Network
       @map = {}
       @reserved = {}
 
-      @commits.each_with_index do |c,i|
+      @commits.each_with_index do |c, i|
         c.time = i
         days[i] = c.committed_date
         @map[c.id] = c
@@ -116,7 +116,7 @@ module Network
     end
 
     def commits_sort_by_ref
-      @commits.sort do |a,b|
+      @commits.sort do |a, b|
         if include_ref?(a)
           -1
         elsif include_ref?(b)

app/models/project.rb

@@ -24,8 +24,12 @@ class Project < ActiveRecord::Base
   default_value_for :wiki_enabled, gitlab_config_features.wiki
   default_value_for :snippets_enabled, gitlab_config_features.snippets
   default_value_for :container_registry_enabled, gitlab_config_features.container_registry
+  default_value_for(:repository_storage) { current_application_settings.repository_storage }
   default_value_for(:shared_runners_enabled) { current_application_settings.shared_runners_enabled }
 
+  after_create :ensure_dir_exist
+  after_save :ensure_dir_exist, if: :namespace_id_changed?
+
   # set last_activity_at to the same as created_at
   after_create :set_last_activity_at
   def set_last_activity_at
@@ -81,6 +85,7 @@ class Project < ActiveRecord::Base
   has_one :jira_service, dependent: :destroy
   has_one :redmine_service, dependent: :destroy
   has_one :custom_issue_tracker_service, dependent: :destroy
+  has_one :bugzilla_service, dependent: :destroy
   has_one :gitlab_issue_tracker_service, dependent: :destroy, inverse_of: :project
   has_one :external_wiki_service, dependent: :destroy
 
@@ -164,6 +169,9 @@ class Project < ActiveRecord::Base
   validate :visibility_level_allowed_by_group
   validate :visibility_level_allowed_as_fork
   validate :check_wiki_path_conflict
+  validates :repository_storage,
+    presence: true,
+    inclusion: { in: ->(_object) { Gitlab.config.repositories.storages.keys } }
 
   add_authentication_token_field :runners_token
   before_save :ensure_runners_token
@@ -375,6 +383,10 @@ class Project < ActiveRecord::Base
     end
   end
 
+  def repository_storage_path
+    Gitlab.config.repositories.storages[repository_storage]
+  end
+
   def team
     @team ||= ProjectTeam.new(self)
   end
@@ -841,12 +853,12 @@ class Project < ActiveRecord::Base
       raise Exception.new('Project cannot be renamed, because tags are present in its container registry')
     end
 
-    if gitlab_shell.mv_repository(old_path_with_namespace, new_path_with_namespace)
+    if gitlab_shell.mv_repository(repository_storage_path, old_path_with_namespace, new_path_with_namespace)
       # If repository moved successfully we need to send update instructions to users.
       # However we cannot allow rollback since we moved repository
       # So we basically we mute exceptions in next actions
       begin
-        gitlab_shell.mv_repository("#{old_path_with_namespace}.wiki", "#{new_path_with_namespace}.wiki")
+        gitlab_shell.mv_repository(repository_storage_path, "#{old_path_with_namespace}.wiki", "#{new_path_with_namespace}.wiki")
         send_move_instructions(old_path_with_namespace)
         reset_events_cache
 
@@ -987,7 +999,7 @@ class Project < ActiveRecord::Base
   def create_repository
     # Forked import is handled asynchronously
     unless forked?
-      if gitlab_shell.add_repository(path_with_namespace)
+      if gitlab_shell.add_repository(repository_storage_path, path_with_namespace)
         repository.after_create
         true
       else
@@ -1139,4 +1151,8 @@ class Project < ActiveRecord::Base
     _, status = Gitlab::Popen.popen(%W(find #{export_path} -not -path #{export_path} -delete))
     status.zero?
   end
+
+  def ensure_dir_exist
+    gitlab_shell.add_namespace(repository_storage_path, namespace.path)
+  end
 end

app/models/project_import_data.rb

@@ -7,6 +7,7 @@ class ProjectImportData < ActiveRecord::Base
                  marshal: true,
                  encode: true,
                  mode: :per_attribute_iv_and_salt,
+                 insecure_mode: true,
                  algorithm: 'aes-256-cbc'
 
   serialize :data, JSON

app/models/project_services/bugzilla_service.rb

+class BugzillaService < IssueTrackerService
+
+  prop_accessor :title, :description, :project_url, :issues_url, :new_issue_url
+
+  def title
+    if self.properties && self.properties['title'].present?
+      self.properties['title']
+    else
+      'Bugzilla'
+    end
+  end
+
+  def description
+    if self.properties && self.properties['description'].present?
+      self.properties['description']
+    else
+      'Bugzilla issue tracker'
+    end
+  end
+
+  def to_param
+    'bugzilla'
+  end
+
+end

app/models/project_services/custom_issue_tracker_service.rb

@@ -32,7 +32,4 @@ class CustomIssueTrackerService < IssueTrackerService
     ]
   end
 
-  def initialize_properties
-    self.properties = {} if properties.nil?
-  end
 end

app/models/project_services/hipchat_service.rb

@@ -106,7 +106,7 @@ class HipchatService < Service
     else
       message << "pushed to #{ref_type} <a href=\""\
                   "#{project.web_url}/commits/#{CGI.escape(ref)}\">#{ref}</a> "
-      message << "of <a href=\"#{project.web_url}\">#{project.name_with_namespace.gsub!(/\s/,'')}</a> "
+      message << "of <a href=\"#{project.web_url}\">#{project.name_with_namespace.gsub!(/\s/, '')}</a> "
       message << "(<a href=\"#{project.web_url}/compare/#{before}...#{after}\">Compare changes</a>)"
 
       push[:commits].take(MAX_COMMITS).each do |commit|

app/models/project_services/jira_service.rb

@@ -124,7 +124,7 @@ class JiraService < IssueTrackerService
 
   def build_api_url_from_project_url
     server = URI(project_url)
-    default_ports = [["http",80],["https",443]].include?([server.scheme,server.port])
+    default_ports = [["http", 80], ["https", 443]].include?([server.scheme, server.port])
     server_url = "#{server.scheme}://#{server.host}"
     server_url.concat(":#{server.port}") unless default_ports
     "#{server_url}/rest/api/#{DEFAULT_API_VERSION}"

app/models/project_team.rb

@@ -137,20 +137,10 @@ class ProjectTeam
   def max_member_access(user_id)
     access = []
 
-    project.members.non_request.each do |member|
-      if member.user_id == user_id
-        access << member.access_field if member.access_field
-        break
-      end
-    end
+    access += project.members.non_request.where(user_id: user_id).has_access.pluck(:access_level)
 
     if group
-      group.members.non_request.each do |member|
-        if member.user_id == user_id
-          access << member.access_field if member.access_field
-          break
-        end
-      end
+      access += group.members.non_request.where(user_id: user_id).has_access.pluck(:access_level)
     end
 
     if project.invited_groups.any? && project.allowed_to_share_with_group?

app/models/project_wiki.rb

@@ -159,7 +159,7 @@ class ProjectWiki
   private
 
   def init_repo(path_with_namespace)
-    gitlab_shell.add_repository(path_with_namespace)
+    gitlab_shell.add_repository(project.repository_storage_path, path_with_namespace)
   end
 
   def commit_details(action, message = nil, title = nil)
@@ -173,7 +173,7 @@ class ProjectWiki
   end
 
   def path_to_repo
-    @path_to_repo ||= File.join(Gitlab.config.gitlab_shell.repos_path, "#{path_with_namespace}.git")
+    @path_to_repo ||= File.join(project.repository_storage_path, "#{path_with_namespace}.git")
   end
 
   def update_project_activity

app/models/repository.rb

@@ -39,7 +39,7 @@ class Repository
   # Return absolute path to repository
   def path_to_repo
     @path_to_repo ||= File.expand_path(
-      File.join(Gitlab.config.gitlab_shell.repos_path, path_with_namespace + ".git")
+      File.join(@project.repository_storage_path, path_with_namespace + ".git")
     )
   end
 
@@ -246,24 +246,26 @@ class Repository
     end
   end
 
+  # Keys for data that can be affected for any commit push.
   def cache_keys
-    %i(size branch_names tag_names branch_count tag_count commit_count
+    %i(size commit_count
        readme version contribution_guide changelog
        license_blob license_key gitignore)
   end
 
+  # Keys for data on branch/tag operations.
+  def cache_keys_for_branches_and_tags
+    %i(branch_names tag_names branch_count tag_count)
+  end
+
   def build_cache
-    cache_keys.each do |key|
+    (cache_keys + cache_keys_for_branches_and_tags).each do |key|
       unless cache.exist?(key)
         send(key)
       end
     end
   end
 
-  def expire_gitignore
-    cache.expire(:gitignore)
-  end
-
   def expire_tags_cache
     cache.expire(:tag_names)
     @tags = nil
@@ -286,8 +288,6 @@ class Repository
     # This ensures this particular cache is flushed after the first commit to a
     # new repository.
     expire_emptiness_caches if empty?
-    expire_branch_count_cache
-    expire_tag_count_cache
   end
 
   def expire_branch_cache(branch_name = nil)
@@ -978,6 +978,10 @@ class Repository
     raw_repository.ls_files(actual_ref)
   end
 
+  def gitattribute(path, name)
+    raw_repository.attributes(path)[name]
+  end
+
   def copy_gitattributes(ref)
     actual_ref = ref || root_ref
     begin

app/models/service.rb

@@ -170,6 +170,7 @@ class Service < ActiveRecord::Base
       bamboo
       buildkite
       builds_email
+      bugzilla
       campfire
       custom_issue_tracker
       drone_ci

app/models/snippet.rb

@@ -20,6 +20,7 @@ class Snippet < ActiveRecord::Base
     length: { within: 0..255 },
     format: { with: Gitlab::Regex.file_name_regex,
               message: Gitlab::Regex.file_name_regex_message }
+
   validates :content, presence: true
   validates :visibility_level, inclusion: { in: Gitlab::VisibilityLevel.values }
 
@@ -81,6 +82,11 @@ class Snippet < ActiveRecord::Base
     0
   end
 
+  # alias for compatibility with blobs and highlighting
+  def path
+    file_name
+  end
+
   def name
     file_name
   end

app/models/user.rb

@@ -25,6 +25,7 @@ class User < ActiveRecord::Base
   attr_encrypted :otp_secret,
     key:       Gitlab::Application.config.secret_key_base,
     mode:      :per_attribute_iv_and_salt,
+    insecure_mode: true,
     algorithm: 'aes-256-cbc'
 
   devise :two_factor_authenticatable,
@@ -763,7 +764,7 @@ class User < ActiveRecord::Base
 
     unless email_domains.blank?
       match_found = email_domains.any? do |domain|
-        escaped = Regexp.escape(domain).gsub('\*','.*?')
+        escaped = Regexp.escape(domain).gsub('\*', '.*?')
         regexp = Regexp.new "^#{escaped}$", Regexp::IGNORECASE
         email_domain = Mail::Address.new(self.email).domain
         email_domain =~ regexp

app/services/projects/destroy_service.rb

@@ -51,13 +51,13 @@ module Projects
       return true if params[:skip_repo] == true
 
       # There is a possibility project does not have repository or wiki
-      return true unless gitlab_shell.exists?(path + '.git')
+      return true unless gitlab_shell.exists?(project.repository_storage_path, path + '.git')
 
       new_path = removal_path(path)
 
-      if gitlab_shell.mv_repository(path, new_path)
+      if gitlab_shell.mv_repository(project.repository_storage_path, path, new_path)
         log_info("Repository \"#{path}\" moved to \"#{new_path}\"")
-        GitlabShellWorker.perform_in(5.minutes, :remove_repository, new_path)
+        GitlabShellWorker.perform_in(5.minutes, :remove_repository, project.repository_storage_path, new_path)
       else
         false
       end

app/services/projects/housekeeping_service.rb

@@ -24,7 +24,7 @@ module Projects
     def execute
       raise LeaseTaken unless try_obtain_lease
 
-      GitlabShellOneShotWorker.perform_async(:gc, @project.path_with_namespace)
+      GitlabShellOneShotWorker.perform_async(:gc, @project.repository_storage_path, @project.path_with_namespace)
     ensure
       Gitlab::Metrics.measure(:reset_pushes_since_gc) do
         @project.update_column(:pushes_since_gc, 0)

app/services/projects/import_service.rb

@@ -42,7 +42,7 @@ module Projects
 
     def import_repository
       begin
-        gitlab_shell.import_repository(project.path_with_namespace, project.import_url)
+        gitlab_shell.import_repository(project.repository_storage_path, project.path_with_namespace, project.import_url)
       rescue Gitlab::Shell::Error => e
         raise Error,  "Error importing repository #{project.import_url} into #{project.path_with_namespace} - #{e.message}"
       end

app/services/projects/transfer_service.rb

@@ -50,12 +50,12 @@ module Projects
         project.send_move_instructions(old_path)
 
         # Move main repository
-        unless gitlab_shell.mv_repository(old_path, new_path)
+        unless gitlab_shell.mv_repository(project.repository_storage_path, old_path, new_path)
           raise TransferError.new('Cannot move project')
         end
 
         # Move wiki repo also if present
-        gitlab_shell.mv_repository("#{old_path}.wiki", "#{new_path}.wiki")
+        gitlab_shell.mv_repository(project.repository_storage_path, "#{old_path}.wiki", "#{new_path}.wiki")
 
         # clear project cached events
         project.reset_events_cache

app/uploaders/lfs_object_uploader.rb

@@ -4,7 +4,7 @@ class LfsObjectUploader < CarrierWave::Uploader::Base
   storage :file
 
   def store_dir
-    "#{Gitlab.config.lfs.storage_path}/#{model.oid[0,2]}/#{model.oid[2,2]}"
+    "#{Gitlab.config.lfs.storage_path}/#{model.oid[0, 2]}/#{model.oid[2, 2]}"
   end
 
   def cache_dir

app/views/admin/application_settings/_form.html.haml

@@ -15,7 +15,7 @@
       = f.label :default_snippet_visibility, class: 'control-label col-sm-2'
       .col-sm-10
         = render('shared/visibility_radios', model_method: :default_snippet_visibility, form: f, selected_level: @application_setting.default_snippet_visibility, form_model: ProjectSnippet.new)
-    .form-group.group-visibility-level-holder
+    .form-group.project-visibility-level-holder
       = f.label :default_group_visibility, class: 'control-label col-sm-2'
       .col-sm-10
         = render('shared/visibility_radios', model_method: :default_group_visibility, form: f, selected_level: @application_setting.default_group_visibility, form_model: Group.new)
@@ -310,6 +310,15 @@
       .col-sm-10
         = f.text_field :sentry_dsn, class: 'form-control'
 
+  %fieldset
+    %legend Repository Storage
+    .form-group
+      = f.label :repository_storage, 'Storage path for new projects', class: 'control-label col-sm-2'
+      .col-sm-10
+        = f.select :repository_storage, repository_storage_options_for_select, {}, class: 'form-control'
+        .help-block
+          You can manage the repository storage paths in your gitlab.yml configuration file
+
   %fieldset
     %legend Repository Checks
     .form-group

app/views/admin/background_jobs/_head.html.haml

 .nav-links.sub-nav
   %ul{ class: (container_class) }
+    = nav_link(controller: :system_info) do
+      = link_to admin_system_info_path, title: 'System Info' do
+        %span
+          System Info
     = nav_link(controller: :background_jobs) do
       = link_to admin_background_jobs_path, title: 'Background Jobs' do
         %span

app/views/admin/background_jobs/show.html.haml

@@ -2,7 +2,7 @@
 - page_title "Background Jobs"
 = render 'admin/background_jobs/head'
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   %h3.page-title Background Jobs
   %p.light GitLab uses #{link_to "sidekiq", "http://sidekiq.org/"} library for async job processing
 

app/views/admin/builds/index.html.haml

 - @no_container = true
 = render "admin/dashboard/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
 
   .top-area
     %ul.nav-links

app/views/admin/dashboard/index.html.haml

 - @no_container = true
 = render "admin/dashboard/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .admin-dashboard.prepend-top-default
     .row
       .col-md-4