Merge remote-tracking branch 'upstream/master' into fix-download-artifacts-button-link

* upstream/master: (80 commits)
  Contributing via GH no longer encouraged.
  Update README.md to really fix icon
  Update README.md to fix icon
  Allow to set request_access_enabled for groups and projects using API
  Restrict last_activity_at updates to one per hour
  Solve code review comments
  Properly support Gitlab::Auth::Result
  Revert all changes introduced by https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6043
  Move Gitlab::Auth.Result to separate file
  Added CHANGELOG
  Fix spec failures
  Support pushing via SSH
  Fix specs failures
  Don't leak build tokens in build logs
  Fix permissions for creating container images
  Fix old CHANGELOG entries
  Add linting for duplicate CHANGELOG versions (!6039)
  Ensure validation messages are shown within the milestone form
  Fix validation regexs (+1 squashed commit) Squashed commits: [f9a9315] Use : to test invalid environment name
  Fix scope of the CI config key nodes in jobs entry
  ...

CHANGELOG

@@ -5,7 +5,9 @@ v 8.12.0 (unreleased)
   - Only check :can_resolve permission if the note is resolvable
   - Bump fog-aws to v0.11.0 to support ap-south-1 region
   - Add ability to fork to a specific namespace using API. (ritave)
+  - Allow to set request_access_enabled for groups and projects
   - Cleanup misalignments in Issue list view !6206
+  - Only create a protected branch upon a push to a new branch if a rule for that branch doesn't exist
   - Prune events older than 12 months. (ritave)
   - Prepend blank line to `Closes` message on merge request linked to issue (lukehowell)
   - Fix issues/merge-request templates dropdown for forked projects
@@ -13,19 +15,25 @@ v 8.12.0 (unreleased)
   - Update gitlab shell secret file also when it is empty. !3774 (glensc)
   - Give project selection dropdowns responsive width, make non-wrapping.
   - Make push events have equal vertical spacing.
+  - API: Ensure invitees are not returned in Members API.
   - Add two-factor recovery endpoint to internal API !5510
   - Pass the "Remember me" value to the U2F authentication form
+  - Only update projects.last_activity_at once per hour when creating a new event
   - Remove vendor prefixes for linear-gradient CSS (ClemMakesApps)
   - Move pushes_since_gc from the database to Redis
   - Add font color contrast to external label in admin area (ClemMakesApps)
   - Change logo animation to CSS (ClemMakesApps)
   - Instructions for enabling Git packfile bitmaps !6104
   - Use Search::GlobalService.new in the `GET /projects/search/:query` endpoint
+  - Fix long comments in diffs messing with table width
   - Fix pagination on user snippets page
+  - Run CI builds with the permissions of users !5735
   - Fix sorting of issues in API
   - Fix download artifacts button links !6407
   - Sort project variables by key. !6275 (Diego Souza)
   - Ensure specs on sorting of issues in API are deterministic on MySQL
+  - Added ability to use predefined CI variables for environment name
+  - Added ability to specify URL in environment configuration in gitlab-ci.yml
   - Escape search term before passing it to Regexp.new !6241 (winniehell)
   - Fix pinned sidebar behavior in smaller viewports !6169
   - Fix file permissions change when updating a file on the Gitlab UI !5979
@@ -51,6 +59,7 @@ v 8.12.0 (unreleased)
   - Add hover color to emoji icon (ClemMakesApps)
   - Increase ci_builds artifacts_size column to 8-byte integer to allow larger files
   - Add textarea autoresize after comment (ClemMakesApps)
+  - Do not write SSH public key 'comments' to authorized_keys !6381
   - Refresh todos count cache when an Issue/MR is deleted
   - Fix branches page dropdown sort alignment (ClemMakesApps)
   - Hides merge request button on branches page is user doesn't have permissions
@@ -140,6 +149,7 @@ v 8.12.0 (unreleased)
   - Use default clone protocol on "check out, review, and merge locally" help page URL
   - API for Ci Lint !5953 (Katarzyna Kobierska Urszula Budziszewska)
   - Allow bulk update merge requests from merge requests index page
+  - Ensure validation messages are shown within the milestone form
   - Add notification_settings API calls !5632 (mahcsig)
   - Remove duplication between project builds and admin builds view !5680 (Katarzyna Kobierska Ula Budziszewska)
   - Fix URLs with anchors in wiki !6300 (houqp)
@@ -148,6 +158,7 @@ v 8.12.0 (unreleased)
   - Fix Gitlab::Popen.popen thread-safety issue
   - Add specs to removing project (Katarzyna Kobierska Ula Budziszewska)
   - Clean environment variables when running git hooks
+  - Fix Import/Export issues importing protected branches and some specific models
   - Fix non-master branch readme display in tree view
 
 v 8.11.6
@@ -595,6 +606,7 @@ v 8.10.0
   - Export and import avatar as part of project import/export
   - Fix migration corrupting import data for old version upgrades
   - Show tooltip on GitLab export link in new project page
+  - Fix import_data wrongly saved as a result of an invalid import_url !5206
 
 v 8.9.9
   - Exclude some pending or inactivated rows in Member scopes
@@ -615,12 +627,6 @@ v 8.9.6
   - Keeps issue number when importing from Gitlab.com
   - Add Pending tab for Builds (Katarzyna Kobierska, Urszula Budziszewska)
 
-v 8.9.7 (unreleased)
-  - Fix import_data wrongly saved as a result of an invalid import_url
-
-v 8.9.6
-  - Fix importing of events under notes for GitLab projects
-
 v 8.9.5
   - Add more debug info to import/export and memory killer. !5108
   - Fixed avatar alignment in new MR view. !5095
@@ -1886,7 +1892,7 @@ v 8.1.3
   - Use issue editor as cross reference comment author when issue is edited with a new mention
   - Add Facebook authentication
 
-v 8.1.1
+v 8.1.2
   - Fix cloning Wiki repositories via HTTP (Stan Hu)
   - Add migration to remove satellites directory
   - Fix specific runners visibility

GITLAB_WORKHORSE_VERSION

-0.8.1
+0.8.2

README.md

@@ -3,10 +3,11 @@
 [![build status](https://gitlab.com/gitlab-org/gitlab-ce/badges/master/build.svg)](https://gitlab.com/gitlab-org/gitlab-ce/commits/master)
 [![coverage report](https://gitlab.com/gitlab-org/gitlab-ce/badges/master/coverage.svg?job=coverage)](https://gitlab.com/gitlab-org/gitlab-ce/commits/master)
 [![Code Climate](https://codeclimate.com/github/gitlabhq/gitlabhq.svg)](https://codeclimate.com/github/gitlabhq/gitlabhq)
+[![Core Infrastructure Initiative Best Practices](https://bestpractices.coreinfrastructure.org/projects/42/badge)](https://bestpractices.coreinfrastructure.org/projects/42)
 
 ## Canonical source
 
-The source of GitLab Community Edition is [hosted on GitLab.com](https://gitlab.com/gitlab-org/gitlab-ce/) and there are mirrors to make [contributing](CONTRIBUTING.md) as easy as possible.
+The cannonical source of GitLab Community Edition is [hosted on GitLab.com](https://gitlab.com/gitlab-org/gitlab-ce/).
 
 ## Open source software to collaborate on code
 

app/assets/javascripts/build.js

@@ -27,10 +27,11 @@
       $(document).off('click', '.js-sidebar-build-toggle').on('click', '.js-sidebar-build-toggle', this.toggleSidebar);
       $(window).off('resize.build').on('resize.build', this.hideSidebar);
       $(document).off('click', '.stage-item').on('click', '.stage-item', this.updateDropdown);
+      $('#js-build-scroll > a').off('click').on('click', this.stepTrace);
       this.updateArtifactRemoveDate();
       if ($('#build-trace').length) {
         this.getInitialBuildTrace();
-        this.initScrollButtonAffix();
+        this.initScrollButtons();
       }
       if (this.build_status === "running" || this.build_status === "pending") {
         $('#autoscroll-button').on('click', function() {
@@ -106,7 +107,7 @@
       }
     };
 
-    Build.prototype.initScrollButtonAffix = function() {
+    Build.prototype.initScrollButtons = function() {
       var $body, $buildScroll, $buildTrace;
       $buildScroll = $('#js-build-scroll');
       $body = $('body');
@@ -165,6 +166,14 @@
       this.populateJobs(stage);
     };
 
+    Build.prototype.stepTrace = function(e) {
+      e.preventDefault();
+      $currentTarget = $(e.currentTarget);
+      $.scrollTo($currentTarget.attr('href'), {
+        offset: -($('.navbar-gitlab').outerHeight() + $('.layout-nav').outerHeight())
+      });
+    };
+
     return Build;
 
   })();

app/assets/javascripts/gl_dropdown.js

@@ -607,13 +607,15 @@
           selectedObject = this.renderedData[selectedIndex];
         }
       }
+      field = [];
+      fieldName = typeof this.options.fieldName === 'function' ? this.options.fieldName(selectedObject) : this.options.fieldName;
       value = this.options.id ? this.options.id(selectedObject, el) : selectedObject.id;
       if (isInput) {
         field = $(this.el);
-      } else {
-        field = this.dropdown.parent().find("input[name='" + fieldName + "'][value='" + escape(value) + "']");
+      } else if(value) {
+        field = this.dropdown.parent().find("input[name='" + fieldName + "'][value='" + value.toString().replace(/'/g, '\\\'') + "']");
       }
-      if (el.hasClass(ACTIVE_CLASS)) {
+      if (field.length && el.hasClass(ACTIVE_CLASS)) {
         el.removeClass(ACTIVE_CLASS);
         if (isInput) {
           field.val('');
@@ -623,7 +625,7 @@
       } else if (el.hasClass(INDETERMINATE_CLASS)) {
         el.addClass(ACTIVE_CLASS);
         el.removeClass(INDETERMINATE_CLASS);
-        if (value == null) {
+        if (field.length && value == null) {
           field.remove();
         }
         if (!field.length && fieldName) {
@@ -636,7 +638,7 @@
             this.dropdown.parent().find("input[name='" + fieldName + "']").remove();
           }
         }
-        if (value == null) {
+        if (field.length && value == null) {
           field.remove();
         }
         // Toggle active class for the tick mark
@@ -644,7 +646,7 @@
         if (value != null) {
           if (!field.length && fieldName) {
             this.addInput(fieldName, value, selectedObject);
-          } else {
+          } else if (field.length) {
             field.val(value).trigger('change');
           }
         }
@@ -794,4 +796,4 @@
     });
   };
 
-}).call(this);
+}).call(this);
\ No newline at end of file

app/assets/javascripts/labels_select.js

@@ -166,7 +166,7 @@
                 instance.addInput(this.fieldName, label.id);
               }
             }
-            if ($form.find("input[type='hidden'][name='" + ($dropdown.data('fieldName')) + "'][value='" + escape(this.id(label)) + "']").length) {
+            if (this.id(label) && $form.find("input[type='hidden'][name='" + ($dropdown.data('fieldName')) + "'][value='" + this.id(label).toString().replace(/'/g, '\\\'') + "']").length) {
               selectedClass.push('is-active');
             }
             if ($dropdown.hasClass('js-multiselect') && removesAll) {

app/assets/stylesheets/pages/diff.scss

@@ -68,6 +68,11 @@
       border-collapse: separate;
       margin: 0;
       padding: 0;
+      table-layout: fixed;
+
+      .diff-line-num {
+        width: 50px;
+      }
 
       .line_holder td {
         line-height: $code_line_height;
@@ -98,10 +103,6 @@
     }
 
     tr.line_holder.parallel {
-      .old_line, .new_line {
-        min-width: 50px;
-      }
-
       td.line_content.parallel {
         width: 46%;
       }

app/controllers/jwt_controller.rb

@@ -11,7 +11,10 @@ class JwtController < ApplicationController
     service = SERVICES[params[:service]]
     return head :not_found unless service
 
-    result = service.new(@project, @user, auth_params).execute
+    @authentication_result ||= Gitlab::Auth::Result.new
+
+    result = service.new(@authentication_result.project, @authentication_result.actor, auth_params).
+      execute(authentication_abilities: @authentication_result.authentication_abilities)
 
     render json: result, status: result[:http_status]
   end
@@ -20,30 +23,23 @@ class JwtController < ApplicationController
 
   def authenticate_project_or_user
     authenticate_with_http_basic do |login, password|
-      # if it's possible we first try to authenticate project with login and password
-      @project = authenticate_project(login, password)
-      return if @project
-
-      @user = authenticate_user(login, password)
-      return if @user
+      @authentication_result = Gitlab::Auth.find_for_git_client(login, password, project: nil, ip: request.ip)
 
-      render_403
+      render_403 unless @authentication_result.success? &&
+        (@authentication_result.actor.nil? || @authentication_result.actor.is_a?(User))
     end
+  rescue Gitlab::Auth::MissingPersonalTokenError
+    render_missing_personal_token
   end
 
-  def auth_params
-    params.permit(:service, :scope, :account, :client_id)
+  def render_missing_personal_token
+    render plain: "HTTP Basic: Access denied\n" \
+                  "You have 2FA enabled, please use a personal access token for Git over HTTP.\n" \
+                  "You can generate one at #{profile_personal_access_tokens_url}",
+           status: 401
   end
 
-  def authenticate_project(login, password)
-    if login == 'gitlab-ci-token'
-      Project.with_builds_enabled.find_by(runners_token: password)
-    end
-  end
-
-  def authenticate_user(login, password)
-    user = Gitlab::Auth.find_with_user_password(login, password)
-    Gitlab::Auth.rate_limit!(request.ip, success: user.present?, login: login)
-    user
+  def auth_params
+    params.permit(:service, :scope, :account, :client_id)
   end
 end

app/controllers/projects/builds_controller.rb

@@ -35,7 +35,11 @@ class Projects::BuildsController < Projects::ApplicationController
     respond_to do |format|
       format.html
       format.json do
-        render json: @build.to_json(methods: :trace_html)
+        render json: {
+          id: @build.id,
+          status: @build.status,
+          trace_html: @build.trace_html
+        }
       end
     end
   end

app/controllers/projects/git_http_client_controller.rb

@@ -4,7 +4,11 @@ class Projects::GitHttpClientController < Projects::ApplicationController
   include ActionController::HttpAuthentication::Basic
   include KerberosSpnegoHelper
 
-  attr_reader :user
+  attr_reader :authentication_result
+
+  delegate :actor, :authentication_abilities, to: :authentication_result, allow_nil: true
+
+  alias_method :user, :actor
 
   # Git clients will not know what authenticity token to send along
   skip_before_action :verify_authenticity_token
@@ -15,32 +19,25 @@ class Projects::GitHttpClientController < Projects::ApplicationController
   private
 
   def authenticate_user
+    @authentication_result = Gitlab::Auth::Result.new
+
     if project && project.public? && download_request?
       return # Allow access
     end
 
     if allow_basic_auth? && basic_auth_provided?
       login, password = user_name_and_password(request)
-      auth_result = Gitlab::Auth.find_for_git_client(login, password, project: project, ip: request.ip)
-
-      if auth_result.type == :ci && download_request?
-        @ci = true
-      elsif auth_result.type == :oauth && !download_request?
-        # Not allowed
-      elsif auth_result.type == :missing_personal_token
-        render_missing_personal_token
-        return # Render above denied access, nothing left to do
-      else
-        @user = auth_result.user
-      end
 
-      if ci? || user
+      if handle_basic_authentication(login, password)
         return # Allow access
       end
     elsif allow_kerberos_spnego_auth? && spnego_provided?
-      @user = find_kerberos_user
+      user = find_kerberos_user
 
       if user
+        @authentication_result = Gitlab::Auth::Result.new(
+          user, nil, :kerberos, Gitlab::Auth.full_authentication_abilities)
+
         send_final_spnego_response
         return # Allow access
       end
@@ -48,6 +45,8 @@ class Projects::GitHttpClientController < Projects::ApplicationController
 
     send_challenges
     render plain: "HTTP Basic: Access denied\n", status: 401
+  rescue Gitlab::Auth::MissingPersonalTokenError
+    render_missing_personal_token
   end
 
   def basic_auth_provided?
@@ -114,8 +113,39 @@ class Projects::GitHttpClientController < Projects::ApplicationController
     render plain: 'Not Found', status: :not_found
   end
 
+  def handle_basic_authentication(login, password)
+    @authentication_result = Gitlab::Auth.find_for_git_client(
+      login, password, project: project, ip: request.ip)
+
+    return false unless @authentication_result.success?
+
+    if download_request?
+      authentication_has_download_access?
+    else
+      authentication_has_upload_access?
+    end
+  end
+
   def ci?
-    @ci.present?
+    authentication_result.ci? &&
+      authentication_project &&
+      authentication_project == project
+  end
+
+  def authentication_has_download_access?
+    has_authentication_ability?(:download_code) || has_authentication_ability?(:build_download_code)
+  end
+
+  def authentication_has_upload_access?
+    has_authentication_ability?(:push_code)
+  end
+
+  def has_authentication_ability?(capability)
+    (authentication_abilities || []).include?(capability)
+  end
+
+  def authentication_project
+    authentication_result.project
   end
 
   def verify_workhorse_api!

app/controllers/projects/git_http_controller.rb

@@ -86,7 +86,7 @@ class Projects::GitHttpController < Projects::GitHttpClientController
   end
 
   def access
-    @access ||= Gitlab::GitAccess.new(user, project, 'http')
+    @access ||= Gitlab::GitAccess.new(user, project, 'http', authentication_abilities: authentication_abilities)
   end
 
   def access_check

app/helpers/lfs_helper.rb

@@ -25,13 +25,21 @@ module LfsHelper
   def lfs_download_access?
     return false unless project.lfs_enabled?
 
-    project.public? || ci? || (user && user.can?(:download_code, project))
+    project.public? || ci? || user_can_download_code? || build_can_download_code?
+  end
+
+  def user_can_download_code?
+    has_authentication_ability?(:download_code) && can?(user, :download_code, project)
+  end
+
+  def build_can_download_code?
+    has_authentication_ability?(:build_download_code) && can?(user, :build_download_code, project)
   end
 
   def lfs_upload_access?
     return false unless project.lfs_enabled?
 
-    user && user.can?(:push_code, project)
+    has_authentication_ability?(:push_code) && can?(user, :push_code, project)
   end
 
   def render_lfs_forbidden

app/models/ci/build.rb

 module Ci
   class Build < CommitStatus
+    include TokenAuthenticatable
+
     belongs_to :runner, class_name: 'Ci::Runner'
     belongs_to :trigger_request, class_name: 'Ci::TriggerRequest'
     belongs_to :erased_by, class_name: 'User'
@@ -23,7 +25,10 @@ module Ci
 
     acts_as_taggable
 
+    add_authentication_token_field :token
+
     before_save :update_artifacts_size, if: :artifacts_file_changed?
+    before_save :ensure_token
     before_destroy { project }
 
     after_create :execute_hooks
@@ -38,6 +43,7 @@ module Ci
         new_build.status = 'pending'
         new_build.runner_id = nil
         new_build.trigger_request_id = nil
+        new_build.token = nil
         new_build.save
       end
 
@@ -79,11 +85,14 @@ module Ci
 
       after_transition any => [:success] do |build|
         if build.environment.present?
-          service = CreateDeploymentService.new(build.project, build.user,
-                                                environment: build.environment,
-                                                sha: build.sha,
-                                                ref: build.ref,
-                                                tag: build.tag)
+          service = CreateDeploymentService.new(
+            build.project, build.user,
+            environment: build.environment,
+            sha: build.sha,
+            ref: build.ref,
+            tag: build.tag,
+            options: build.options[:environment],
+            variables: build.variables)
           service.execute(build)
         end
       end
@@ -173,7 +182,7 @@ module Ci
     end
 
     def repo_url
-      auth = "gitlab-ci-token:#{token}@"
+      auth = "gitlab-ci-token:#{ensure_token!}@"
       project.http_url_to_repo.sub(/^https?:\/\//) do |prefix|
         prefix + auth
       end
@@ -235,12 +244,7 @@ module Ci
     end
 
     def trace
-      trace = raw_trace
-      if project && trace.present? && project.runners_token.present?
-        trace.gsub(project.runners_token, 'xxxxxx')
-      else
-        trace
-      end
+      hide_secrets(raw_trace)
     end
 
     def trace_length
@@ -253,6 +257,7 @@ module Ci
 
     def trace=(trace)
       recreate_trace_dir
+      trace = hide_secrets(trace)
       File.write(path_to_trace, trace)
     end
 
@@ -266,6 +271,8 @@ module Ci
     def append_trace(trace_part, offset)
       recreate_trace_dir
 
+      trace_part = hide_secrets(trace_part)
+
       File.truncate(path_to_trace, offset) if File.exist?(path_to_trace)
       File.open(path_to_trace, 'ab') do |f|
         f.write(trace_part)
@@ -341,12 +348,8 @@ module Ci
       )
     end
 
-    def token
-      project.runners_token
-    end
-
     def valid_token?(token)
-      project.valid_runners_token?(token)
+      self.token && ActiveSupport::SecurityUtils.variable_size_secure_compare(token, self.token)
     end
 
     def has_tags?
@@ -488,5 +491,11 @@ module Ci
 
       pipeline.config_processor.build_attributes(name)
     end
+
+    def hide_secrets(trace)
+      trace = Ci::MaskSecret.mask(trace, project.runners_token) if project
+      trace = Ci::MaskSecret.mask(trace, token)
+      trace
+    end
   end
 end

app/models/ci/pipeline.rb

@@ -2,6 +2,7 @@ module Ci
   class Pipeline < ActiveRecord::Base
     extend Ci::Model
     include HasStatus
+    include Importable
 
     self.table_name = 'ci_commits'
 
@@ -12,12 +13,12 @@ module Ci
     has_many :builds, class_name: 'Ci::Build', foreign_key: :commit_id
     has_many :trigger_requests, dependent: :destroy, class_name: 'Ci::TriggerRequest', foreign_key: :commit_id
 
-    validates_presence_of :sha
-    validates_presence_of :ref
-    validates_presence_of :status
-    validate :valid_commit_sha
+    validates_presence_of :sha, unless: :importing?
+    validates_presence_of :ref, unless: :importing?
+    validates_presence_of :status, unless: :importing?
+    validate :valid_commit_sha, unless: :importing?
 
-    after_save :keep_around_commits
+    after_save :keep_around_commits, unless: :importing?
 
     delegate :stages, to: :statuses
 

app/models/environment.rb

@@ -4,6 +4,7 @@ class Environment < ActiveRecord::Base
   has_many :deployments
 
   before_validation :nullify_external_url
+  before_save :set_environment_type
 
   validates :name,
             presence: true,
@@ -26,6 +27,17 @@ class Environment < ActiveRecord::Base
     self.external_url = nil if self.external_url.blank?
   end
 
+  def set_environment_type
+    names = name.split('/')
+
+    self.environment_type =
+      if names.many?
+        names.first
+      else
+        nil
+      end
+  end
+
   def includes_commit?(commit)
     return false unless last_deployment
 

app/models/event.rb

@@ -13,6 +13,8 @@ class Event < ActiveRecord::Base
   LEFT      = 9 # User left project
   DESTROYED = 10
 
+  RESET_PROJECT_ACTIVITY_INTERVAL = 1.hour
+
   delegate :name, :email, to: :author, prefix: true, allow_nil: true
   delegate :title, to: :issue, prefix: true, allow_nil: true
   delegate :title, to: :merge_request, prefix: true, allow_nil: true
@@ -324,8 +326,17 @@ class Event < ActiveRecord::Base
   end
 
   def reset_project_activity
-    if project && Gitlab::ExclusiveLease.new("project:update_last_activity_at:#{project.id}", timeout: 60).try_obtain
-      project.update_column(:last_activity_at, self.created_at)
-    end
+    return unless project
+
+    # Don't even bother obtaining a lock if the last update happened less than
+    # 60 minutes ago.
+    return if project.last_activity_at > RESET_PROJECT_ACTIVITY_INTERVAL.ago
+
+    return unless Gitlab::ExclusiveLease.
+      new("project:update_last_activity_at:#{project.id}",
+          timeout: RESET_PROJECT_ACTIVITY_INTERVAL.to_i).
+      try_obtain
+
+    project.update_column(:last_activity_at, created_at)
   end
 end

app/models/project.rb

@@ -1137,12 +1137,6 @@ class Project < ActiveRecord::Base
     self.runners_token && ActiveSupport::SecurityUtils.variable_size_secure_compare(token, self.runners_token)
   end
 
-  # TODO (ayufan): For now we use runners_token (backward compatibility)
-  # In 8.4 every build will have its own individual token valid for time of build
-  def valid_build_token?(token)
-    self.builds_enabled? && self.runners_token && ActiveSupport::SecurityUtils.variable_size_secure_compare(token, self.runners_token)
-  end
-
   def build_coverage_enabled?
     build_coverage_regex.present?
   end

app/policies/project_policy.rb

@@ -64,6 +64,12 @@ class ProjectPolicy < BasePolicy
     can! :read_deployment
   end
 
+  # Permissions given when an user is team member of a project
+  def team_member_reporter_access!
+    can! :build_download_code
+    can! :build_read_container_image
+  end
+
   def developer_access!
     can! :admin_merge_request
     can! :update_merge_request
@@ -109,6 +115,8 @@ class ProjectPolicy < BasePolicy
     can! :read_commit_status
     can! :read_pipeline
     can! :read_container_image
+    can! :build_download_code
+    can! :build_read_container_image
   end
 
   def owner_access!
@@ -130,10 +138,11 @@ class ProjectPolicy < BasePolicy
   def team_access!(user)
     access = project.team.max_member_access(user.id)
 
-    guest_access!     if access >= Gitlab::Access::GUEST
-    reporter_access!  if access >= Gitlab::Access::REPORTER
-    developer_access! if access >= Gitlab::Access::DEVELOPER
-    master_access!    if access >= Gitlab::Access::MASTER
+    guest_access!                if access >= Gitlab::Access::GUEST
+    reporter_access!             if access >= Gitlab::Access::REPORTER
+    team_member_reporter_access! if access >= Gitlab::Access::REPORTER
+    developer_access!            if access >= Gitlab::Access::DEVELOPER
+    master_access!               if access >= Gitlab::Access::MASTER
   end
 
   def archived_access!

app/services/auth/container_registry_authentication_service.rb

@@ -4,7 +4,9 @@ module Auth
 
     AUDIENCE = 'container_registry'
 
-    def execute
+    def execute(authentication_abilities:)
+      @authentication_abilities = authentication_abilities || []
+
       return error('not found', 404) unless registry.enabled
 
       unless current_user || project
@@ -74,9 +76,9 @@ module Auth
 
       case requested_action
       when 'pull'
-        requested_project == project || can?(current_user, :read_container_image, requested_project)
+        requested_project.public? || build_can_pull?(requested_project) || user_can_pull?(requested_project)
       when 'push'
-        requested_project == project || can?(current_user, :create_container_image, requested_project)
+        build_can_push?(requested_project) || user_can_push?(requested_project)
       else
         false
       end
@@ -85,5 +87,29 @@ module Auth
     def registry
       Gitlab.config.registry
     end
+
+    def build_can_pull?(requested_project)
+      # Build can:
+      # 1. pull from its own project (for ex. a build)
+      # 2. read images from dependent projects if creator of build is a team member
+      @authentication_abilities.include?(:build_read_container_image) &&
+        (requested_project == project || can?(current_user, :build_read_container_image, requested_project))
+    end
+
+    def user_can_pull?(requested_project)
+      @authentication_abilities.include?(:read_container_image) &&
+        can?(current_user, :read_container_image, requested_project)
+    end
+
+    def build_can_push?(requested_project)
+      # Build can push only to the project from which it originates
+      @authentication_abilities.include?(:build_create_container_image) &&
+        requested_project == project
+    end
+
+    def user_can_push?(requested_project)
+      @authentication_abilities.include?(:create_container_image) &&
+        can?(current_user, :create_container_image, requested_project)
+    end
   end
 end

app/services/create_deployment_service.rb

@@ -2,9 +2,7 @@ require_relative 'base_service'
 
 class CreateDeploymentService < BaseService
   def execute(deployable = nil)
-    environment = project.environments.find_or_create_by(
-      name: params[:environment]
-    )
+    environment = find_or_create_environment
 
     project.deployments.create(
       environment: environment,
@@ -15,4 +13,38 @@ class CreateDeploymentService < BaseService
       deployable: deployable
     )
   end
+
+  private
+
+  def find_or_create_environment
+    project.environments.find_or_create_by(name: expanded_name) do |environment|
+      environment.external_url = expanded_url
+    end
+  end
+
+  def expanded_name
+    ExpandVariables.expand(name, variables)
+  end
+
+  def expanded_url
+    return unless url
+
+    @expanded_url ||= ExpandVariables.expand(url, variables)
+  end
+
+  def name
+    params[:environment]
+  end
+
+  def url
+    options[:url]
+  end
+
+  def options
+    params[:options] || {}
+  end
+
+  def variables
+    params[:variables] || []
+  end
 end

app/services/git_push_service.rb

@@ -87,7 +87,7 @@ class GitPushService < BaseService
     project.change_head(branch_name)
 
     # Set protection on the default branch if configured
-    if current_application_settings.default_branch_protection != PROTECTION_NONE
+    if current_application_settings.default_branch_protection != PROTECTION_NONE && !@project.protected_branch?(@project.default_branch)
 
       params = {
         name: @project.default_branch,

app/services/milestones/create_service.rb

@@ -3,7 +3,7 @@ module Milestones
     def execute
       milestone = project.milestones.new(params)
 
-      if milestone.save!
+      if milestone.save
         event_service.open_milestone(milestone, current_user)
       end
 

app/views/shared/issuable/_label_dropdown.html.haml

@@ -12,7 +12,7 @@
 - if params[:label_name].present?
   - if params[:label_name].respond_to?('any?')
     - params[:label_name].each do |label|
-      = hidden_field_tag "label_name[]", u(label), id: nil
+      = hidden_field_tag "label_name[]", label, id: nil
 .dropdown
   %button.dropdown-menu-toggle.js-label-select.js-multiselect{class: classes.join(' '), type: "button", data: dropdown_data}
     %span.dropdown-toggle-text

db/migrate/20160808085531_add_token_to_build.rb

+class AddTokenToBuild < ActiveRecord::Migration
+  include Gitlab::Database::MigrationHelpers
+
+  # Set this constant to true if this migration requires downtime.
+  DOWNTIME = false
+
+  def change
+    add_column :ci_builds, :token, :string
+  end
+end

db/migrate/20160808085602_add_index_for_build_token.rb

+class AddIndexForBuildToken < ActiveRecord::Migration
+  include Gitlab::Database::MigrationHelpers
+
+  # Set this constant to true if this migration requires downtime.
+  DOWNTIME = false
+
+  disable_ddl_transaction!
+
+  def change
+    add_concurrent_index :ci_builds, :token, unique: true
+  end
+end

db/migrate/20160907131111_add_environment_type_to_environments.rb

+class AddEnvironmentTypeToEnvironments < ActiveRecord::Migration
+  include Gitlab::Database::MigrationHelpers
+
+  DOWNTIME = false
+
+  def change
+    add_column :environments, :environment_type, :string
+  end
+end

db/schema.rb

@@ -181,6 +181,7 @@ ActiveRecord::Schema.define(version: 20160913212128) do
     t.string   "when"
     t.text     "yaml_variables"
     t.datetime "queued_at"
+    t.string   "token"
   end
 
   add_index "ci_builds", ["commit_id", "stage_idx", "created_at"], name: "index_ci_builds_on_commit_id_and_stage_idx_and_created_at", using: :btree
@@ -192,6 +193,7 @@ ActiveRecord::Schema.define(version: 20160913212128) do
   add_index "ci_builds", ["project_id"], name: "index_ci_builds_on_project_id", using: :btree
   add_index "ci_builds", ["runner_id"], name: "index_ci_builds_on_runner_id", using: :btree
   add_index "ci_builds", ["status"], name: "index_ci_builds_on_status", using: :btree
+  add_index "ci_builds", ["token"], name: "index_ci_builds_on_token", unique: true, using: :btree
 
   create_table "ci_commits", force: :cascade do |t|
     t.integer  "project_id"
@@ -390,10 +392,11 @@ ActiveRecord::Schema.define(version: 20160913212128) do
 
   create_table "environments", force: :cascade do |t|
     t.integer  "project_id"
-    t.string   "name",         null: false
+    t.string   "name",             null: false
     t.datetime "created_at"
     t.datetime "updated_at"
     t.string   "external_url"
+    t.string   "environment_type"
   end
 
   add_index "environments", ["project_id", "name"], name: "index_environments_on_project_id_and_name", using: :btree

doc/api/groups.md

@@ -84,7 +84,8 @@ Parameters:
     "forks_count": 0,
     "open_issues_count": 3,
     "public_builds": true,
-    "shared_with_groups": []
+    "shared_with_groups": [],
+    "request_access_enabled": false
   }
 ]
 ```
@@ -118,6 +119,7 @@ Example response:
   "visibility_level": 20,
   "avatar_url": null,
   "web_url": "https://gitlab.example.com/groups/twitter",
+  "request_access_enabled": false,
   "projects": [
     {
       "id": 7,
@@ -163,7 +165,8 @@ Example response:
       "forks_count": 0,
       "open_issues_count": 3,
       "public_builds": true,
-      "shared_with_groups": []
+      "shared_with_groups": [],
+      "request_access_enabled": false
     },
     {
       "id": 6,
@@ -209,7 +212,8 @@ Example response:
       "forks_count": 0,
       "open_issues_count": 8,
       "public_builds": true,
-      "shared_with_groups": []
+      "shared_with_groups": [],
+      "request_access_enabled": false
     }
   ],
   "shared_projects": [
@@ -289,6 +293,7 @@ Parameters:
 - `description` (optional) - The group's description
 - `visibility_level` (optional) - The group's visibility. 0 for private, 10 for internal, 20 for public.
 - `lfs_enabled` (optional)      - Enable/disable Large File Storage (LFS) for the projects in this group
+- `request_access_enabled` (optional) - Allow users to request member access.
 
 ## Transfer project to group
 
@@ -319,6 +324,7 @@ PUT /groups/:id
 | `description` | string | no | The description of the group |
 | `visibility_level` | integer | no | The visibility level of the group. 0 for private, 10 for internal, 20 for public. |
 | `lfs_enabled` (optional) | boolean | no | Enable/disable Large File Storage (LFS) for the projects in this group |
+| `request_access_enabled` | boolean | no | Allow users to request member access. |
 
 ```bash
 curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/groups/5?name=Experimental"
@@ -336,6 +342,7 @@ Example response:
   "visibility_level": 10,
   "avatar_url": null,
   "web_url": "http://gitlab.example.com/groups/h5bp",
+  "request_access_enabled": false,
   "projects": [
     {
       "id": 9,
@@ -380,7 +387,8 @@ Example response:
       "forks_count": 0,
       "open_issues_count": 3,
       "public_builds": true,
-      "shared_with_groups": []
+      "shared_with_groups": [],
+      "request_access_enabled": false
     }
   ]
 }

doc/api/projects.md

@@ -85,7 +85,8 @@ Parameters:
     "runners_token": "b8547b1dc37721d05889db52fa2f02",
     "public_builds": true,
     "shared_with_groups": [],
-    "only_allow_merge_if_build_succeeds": false
+    "only_allow_merge_if_build_succeeds": false,
+    "request_access_enabled": false
   },
   {
     "id": 6,
@@ -146,7 +147,8 @@ Parameters:
     "runners_token": "b8547b1dc37721d05889db52fa2f02",
     "public_builds": true,
     "shared_with_groups": [],
-    "only_allow_merge_if_build_succeeds": false
+    "only_allow_merge_if_build_succeeds": false,
+    "request_access_enabled": false
   }
 ]
 ```
@@ -283,7 +285,8 @@ Parameters:
       "group_access_level": 10
     }
   ],
-  "only_allow_merge_if_build_succeeds": false
+  "only_allow_merge_if_build_succeeds": false,
+  "request_access_enabled": false
 }
 ```
 
@@ -453,6 +456,7 @@ Parameters:
 - `public_builds` (optional)
 - `only_allow_merge_if_build_succeeds` (optional)
 - `lfs_enabled` (optional)
+- `request_access_enabled` (optional) - Allow users to request member access.
 
 ### Create project for user
 
@@ -480,6 +484,7 @@ Parameters:
 - `public_builds` (optional)
 - `only_allow_merge_if_build_succeeds` (optional)
 - `lfs_enabled` (optional)
+- `request_access_enabled` (optional) - Allow users to request member access.
 
 ### Edit project
 
@@ -508,6 +513,7 @@ Parameters:
 - `public_builds` (optional)
 - `only_allow_merge_if_build_succeeds` (optional)
 - `lfs_enabled` (optional)
+- `request_access_enabled` (optional) - Allow users to request member access.
 
 On success, method returns 200 with the updated project. If parameters are
 invalid, 400 is returned.
@@ -588,7 +594,8 @@ Example response:
   "star_count": 1,
   "public_builds": true,
   "shared_with_groups": [],
-  "only_allow_merge_if_build_succeeds": false
+  "only_allow_merge_if_build_succeeds": false,
+  "request_access_enabled": false
 }
 ```
 
@@ -655,7 +662,8 @@ Example response:
   "star_count": 0,
   "public_builds": true,
   "shared_with_groups": [],
-  "only_allow_merge_if_build_succeeds": false
+  "only_allow_merge_if_build_succeeds": false,
+  "request_access_enabled": false
 }
 ```
 
@@ -742,7 +750,8 @@ Example response:
   "runners_token": "b8bc4a7a29eb76ea83cf79e4908c2b",
   "public_builds": true,
   "shared_with_groups": [],
-  "only_allow_merge_if_build_succeeds": false
+  "only_allow_merge_if_build_succeeds": false,
+  "request_access_enabled": false
 }
 ```
 
@@ -829,7 +838,8 @@ Example response:
   "runners_token": "b8bc4a7a29eb76ea83cf79e4908c2b",
   "public_builds": true,
   "shared_with_groups": [],
-  "only_allow_merge_if_build_succeeds": false
+  "only_allow_merge_if_build_succeeds": false,
+  "request_access_enabled": false
 }
 ```
 

doc/ci/yaml/README.md

@@ -90,8 +90,7 @@ builds, including deploy builds. This can be an array or a multi-line string.
 
 ### after_script
 
->**Note:**
-Introduced in GitLab 8.7 and requires Gitlab Runner v1.2
+> Introduced in GitLab 8.7 and requires Gitlab Runner v1.2
 
 `after_script` is used to define the command that will be run after for all
 builds. This has to be an array or a multi-line string.
@@ -135,8 +134,7 @@ Alias for [stages](#stages).
 
 ### variables
 
->**Note:**
-Introduced in GitLab Runner v0.5.0.
+> Introduced in GitLab Runner v0.5.0.
 
 GitLab CI allows you to add variables to `.gitlab-ci.yml` that are set in the
 build environment. The variables are stored in the Git repository and are meant
@@ -158,8 +156,7 @@ Variables can be also defined on [job level](#job-variables).
 
 ### cache
 
->**Note:**
-Introduced in GitLab Runner v0.7.0.
+> Introduced in GitLab Runner v0.7.0.
 
 `cache` is used to specify a list of files and directories which should be
 cached between builds.
@@ -220,8 +217,7 @@ will be always present. For implementation details, please check GitLab Runner.
 
 #### cache:key
 
->**Note:**
-Introduced in GitLab Runner v1.0.0.
+> Introduced in GitLab Runner v1.0.0.
 
 The `key` directive allows you to define the affinity of caching
 between jobs, allowing to have a single cache for all jobs,
@@ -531,8 +527,7 @@ The above script will:
 
 #### Manual actions
 
->**Note:**
-Introduced in GitLab 8.10.
+> Introduced in GitLab 8.10.
 
 Manual actions are a special type of job that are not executed automatically;
 they need to be explicitly started by a user. Manual actions can be started
@@ -543,17 +538,16 @@ An example usage of manual actions is deployment to production.
 
 ### environment
 
->**Note:**
-Introduced in GitLab 8.9.
+> Introduced in GitLab 8.9.
 
-`environment` is used to define that a job deploys to a specific environment.
+`environment` is used to define that a job deploys to a specific [environment].
 This allows easy tracking of all deployments to your environments straight from
 GitLab.
 
 If `environment` is specified and no environment under that name exists, a new
 one will be created automatically.
 
-The `environment` name must contain only letters, digits, '-' and '_'. Common
+The `environment` name must contain only letters, digits, '-', '_', '/', '$', '{', '}' and spaces. Common
 names are `qa`, `staging`, and `production`, but you can use whatever name works
 with your workflow.
 
@@ -571,6 +565,35 @@ deploy to production:
 The `deploy to production` job will be marked as doing deployment to
 `production` environment.
 
+#### dynamic environments
+
+> [Introduced][ce-6323] in GitLab 8.12 and GitLab Runner 1.6.
+
+`environment` can also represent a configuration hash with `name` and `url`.
+These parameters can use any of the defined CI [variables](#variables)
+(including predefined, secure variables and `.gitlab-ci.yml` variables).
+
+The common use case is to create dynamic environments for branches and use them
+as review apps.
+
+---
+
+**Example configurations**
+
+```
+deploy as review app:
+  stage: deploy
+  script: ...
+  environment:
+    name: review-apps/$CI_BUILD_REF_NAME
+    url: https://$CI_BUILD_REF_NAME.review.example.com/
+```
+
+The `deploy as review app` job will be marked as deployment to dynamically
+create the `review-apps/branch-name` environment.
+
+This environment should be accessible under `https://branch-name.review.example.com/`.
+
 ### artifacts
 
 >**Notes:**
@@ -638,8 +661,7 @@ be available for download in the GitLab UI.
 
 #### artifacts:name
 
->**Note:**
-Introduced in GitLab 8.6 and GitLab Runner v1.1.0.
+> Introduced in GitLab 8.6 and GitLab Runner v1.1.0.
 
 The `name` directive allows you to define the name of the created artifacts
 archive. That way, you can have a unique name for every archive which could be
@@ -702,8 +724,7 @@ job:
 
 #### artifacts:when
 
->**Note:**
-Introduced in GitLab 8.9 and GitLab Runner v1.3.0.
+> Introduced in GitLab 8.9 and GitLab Runner v1.3.0.
 
 `artifacts:when` is used to upload artifacts on build failure or despite the
 failure.
@@ -728,8 +749,7 @@ job:
 
 #### artifacts:expire_in
 
->**Note:**
-Introduced in GitLab 8.9 and GitLab Runner v1.3.0.
+> Introduced in GitLab 8.9 and GitLab Runner v1.3.0.
 
 `artifacts:expire_in` is used to delete uploaded artifacts after the specified
 time. By default, artifacts are stored on GitLab forever. `expire_in` allows you
@@ -764,8 +784,7 @@ job:
 
 ### dependencies
 
->**Note:**
-Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
+> Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
 
 This feature should be used in conjunction with [`artifacts`](#artifacts) and
 allows you to define the artifacts to pass between different builds.
@@ -839,9 +858,8 @@ job:
 
 ## Git Strategy
 
->**Note:**
-Introduced in GitLab 8.9 as an experimental feature. May change in future
-releases or be removed completely.
+> Introduced in GitLab 8.9 as an experimental feature. May change in future
+  releases or be removed completely.
 
 You can set the `GIT_STRATEGY` used for getting recent application code. `clone`
 is slower, but makes sure you have a clean directory before every build. `fetch`
@@ -863,8 +881,7 @@ variables:
 
 ## Shallow cloning
 
->**Note:**
-Introduced in GitLab 8.9 as an experimental feature. May change in future
+> Introduced in GitLab 8.9 as an experimental feature. May change in future
 releases or be removed completely.
 
 You can specify the depth of fetching and cloning using `GIT_DEPTH`. This allows
@@ -894,8 +911,7 @@ variables:
 
 ## Hidden keys
 
->**Note:**
-Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
+> Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
 
 Keys that start with a dot (`.`) will be not processed by GitLab CI. You can
 use this feature to ignore jobs, or use the
@@ -923,8 +939,7 @@ Read more about the various [YAML features](https://learnxinyminutes.com/docs/ya
 
 ### Anchors
 
->**Note:**
-Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
+> Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
 
 YAML also has a handy feature called 'anchors', which let you easily duplicate
 content across your document. Anchors can be used to duplicate/inherit
@@ -1067,3 +1082,5 @@ Visit the [examples README][examples] to see a list of examples using GitLab
 CI with various languages.
 
 [examples]: ../examples/README.md
+[ce-6323]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6323
+[environment]: ../environments.md

doc/install/installation.md

@@ -400,7 +400,7 @@ If you are not using Linux you may have to run `gmake` instead of
     cd /home/git
     sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-workhorse.git
     cd gitlab-workhorse
-    sudo -u git -H git checkout v0.8.1
+    sudo -u git -H git checkout v0.8.2
     sudo -u git -H make
 
 ### Initialize Database and Activate Advanced Features

doc/update/8.11-to-8.12.md

@@ -82,7 +82,7 @@ GitLab 8.1.
 ```bash
 cd /home/git/gitlab-workhorse
 sudo -u git -H git fetch --all
-sudo -u git -H git checkout v0.8.1
+sudo -u git -H git checkout v0.8.2
 sudo -u git -H make
 ```
 

doc/user/project/settings/import_export.md

@@ -3,8 +3,8 @@
 >**Notes:**
 >
 >  - [Introduced][ce-3050] in GitLab 8.9.
->  - Importing will not be possible if the import instance version is lower
->    than that of the exporter.
+>  - Importing will not be possible if the import instance version differs from
+>    that of the exporter.
 >  - For existing installations, the project import option has to be enabled in
 >    application settings (`/admin/application_settings`) under 'Import sources'.
 >    You will have to be an administrator to enable and use the import functionality.
@@ -17,6 +17,20 @@
 Existing projects running on any GitLab instance or GitLab.com can be exported
 with all their related data and be moved into a new GitLab instance.
 
+## Version history
+
+| GitLab version | Import/Export version |
+| -------- | -------- |
+| 8.12.0 to current  | 0.1.4    |
+| 8.10.3   | 0.1.3    |
+| 8.10.0   | 0.1.2    |
+| 8.9.5    | 0.1.1    |
+| 8.9.0    | 0.1.0    |
+ 
+ > The table reflects what GitLab version we updated the Import/Export version at.
+ > For instance, 8.10.3 and 8.11 will have the same Import/Export version (0.1.3)
+ > and the exports between them will be compatible.
+
 ## Exported contents
 
 The following items will be exported:

lib/api/access_requests.rb

@@ -20,7 +20,7 @@ module API
 
           access_requesters = paginate(source.requesters.includes(:user))
 
-          present access_requesters.map(&:user), with: Entities::AccessRequester, access_requesters: access_requesters
+          present access_requesters.map(&:user), with: Entities::AccessRequester, source: source
         end
 
         # Request access to the group/project

lib/api/entities.rb

@@ -100,22 +100,23 @@ module API
         SharedGroup.represent(project.project_group_links.all, options)
       end
       expose :only_allow_merge_if_build_succeeds
+      expose :request_access_enabled
     end
 
     class Member < UserBasic
       expose :access_level do |user, options|
-        member = options[:member] || options[:members].find { |m| m.user_id == user.id }
+        member = options[:member] || options[:source].members.find_by(user_id: user.id)
         member.access_level
       end
       expose :expires_at do |user, options|
-        member = options[:member] || options[:members].find { |m| m.user_id == user.id }
+        member = options[:member] || options[:source].members.find_by(user_id: user.id)
         member.expires_at
       end
     end
 
     class AccessRequester < UserBasic
       expose :requested_at do |user, options|
-        access_requester = options[:access_requester] || options[:access_requesters].find { |m| m.user_id == user.id }
+        access_requester = options[:access_requester] || options[:source].requesters.find_by(user_id: user.id)
         access_requester.requested_at
       end
     end
@@ -125,6 +126,7 @@ module API
       expose :lfs_enabled?, as: :lfs_enabled
       expose :avatar_url
       expose :web_url
+      expose :request_access_enabled
     end
 
     class GroupDetail < Group

lib/api/groups.rb

@@ -23,18 +23,19 @@ module API
       # Create group. Available only for users who can create groups.
       #
       # Parameters:
-      #   name (required)             - The name of the group
-      #   path (required)             - The path of the group
-      #   description (optional)      - The description of the group
-      #   visibility_level (optional) - The visibility level of the group
-      #   lfs_enabled (optional)      - Enable/disable LFS for the projects in this group
+      #   name (required)                   - The name of the group
+      #   path (required)                   - The path of the group
+      #   description (optional)            - The description of the group
+      #   visibility_level (optional)       - The visibility level of the group
+      #   lfs_enabled (optional)            - Enable/disable LFS for the projects in this group
+      #   request_access_enabled (optional) - Allow users to request member access
       # Example Request:
       #   POST /groups
       post do
         authorize! :create_group
         required_attributes! [:name, :path]
 
-        attrs = attributes_for_keys [:name, :path, :description, :visibility_level, :lfs_enabled]
+        attrs = attributes_for_keys [:name, :path, :description, :visibility_level, :lfs_enabled, :request_access_enabled]
         @group = Group.new(attrs)
 
         if @group.save
@@ -48,18 +49,19 @@ module API
       # Update group. Available only for users who can administrate groups.
       #
       # Parameters:
-      #   id (required)               - The ID of a group
-      #   path (optional)             - The path of the group
-      #   description (optional)      - The description of the group
-      #   visibility_level (optional) - The visibility level of the group
-      #   lfs_enabled (optional)      - Enable/disable LFS for the projects in this group
+      #   id (required)                     - The ID of a group
+      #   path (optional)                   - The path of the group
+      #   description (optional)            - The description of the group
+      #   visibility_level (optional)       - The visibility level of the group
+      #   lfs_enabled (optional)            - Enable/disable LFS for the projects in this group
+      #   request_access_enabled (optional) - Allow users to request member access
       # Example Request:
       #   PUT /groups/:id
       put ':id' do
         group = find_group(params[:id])
         authorize! :admin_group, group
 
-        attrs = attributes_for_keys [:name, :path, :description, :visibility_level, :lfs_enabled]
+        attrs = attributes_for_keys [:name, :path, :description, :visibility_level, :lfs_enabled, :request_access_enabled]
 
         if ::Groups::UpdateService.new(group, current_user, attrs).execute
           present group, with: Entities::GroupDetail

lib/api/internal.rb

@@ -35,6 +35,14 @@ module API
             Project.find_with_namespace(project_path)
           end
         end
+
+        def ssh_authentication_abilities
+          [
+            :read_project,
+            :download_code,
+            :push_code
+          ]
+        end
       end
 
       post "/allowed" do
@@ -51,9 +59,9 @@ module API
 
         access =
           if wiki?
-            Gitlab::GitAccessWiki.new(actor, project, protocol)
+            Gitlab::GitAccessWiki.new(actor, project, protocol, authentication_abilities: ssh_authentication_abilities)
           else
-            Gitlab::GitAccess.new(actor, project, protocol)
+            Gitlab::GitAccess.new(actor, project, protocol, authentication_abilities: ssh_authentication_abilities)
           end
 
         access_status = access.check(params[:action], params[:changes])

lib/api/members.rb

@@ -18,11 +18,11 @@ module API
         get ":id/members" do
           source = find_source(source_type, params[:id])
 
-          members = source.members.includes(:user)
-          members = members.joins(:user).merge(User.search(params[:query])) if params[:query]
-          members = paginate(members)
+          users = source.users
+          users = users.merge(User.search(params[:query])) if params[:query]
+          users = paginate(users)
 
-          present members.map(&:user), with: Entities::Member, members: members
+          present users, with: Entities::Member, source: source
         end
 
         # Get a group/project member

lib/api/projects.rb

@@ -91,8 +91,8 @@ module API
       # Create new project
       #
       # Parameters:
-      #   name (required) - name for new project
-      #   description (optional) - short project description
+      #   name (required)                   - name for new project
+      #   description (optional)            - short project description
       #   issues_enabled (optional)
       #   merge_requests_enabled (optional)
       #   builds_enabled (optional)
@@ -100,33 +100,35 @@ module API
       #   snippets_enabled (optional)
       #   container_registry_enabled (optional)
       #   shared_runners_enabled (optional)
-      #   namespace_id (optional) - defaults to user namespace
-      #   public (optional) - if true same as setting visibility_level = 20
-      #   visibility_level (optional) - 0 by default
+      #   namespace_id (optional)           - defaults to user namespace
+      #   public (optional)                 - if true same as setting visibility_level = 20
+      #   visibility_level (optional)       - 0 by default
       #   import_url (optional)
       #   public_builds (optional)
       #   lfs_enabled (optional)
+      #   request_access_enabled (optional) - Allow users to request member access
       # Example Request
       #   POST /projects
       post do
         required_attributes! [:name]
-        attrs = attributes_for_keys [:name,
-                                     :path,
+        attrs = attributes_for_keys [:builds_enabled,
+                                     :container_registry_enabled,
                                      :description,
+                                     :import_url,
                                      :issues_enabled,
+                                     :lfs_enabled,
                                      :merge_requests_enabled,
-                                     :builds_enabled,
-                                     :wiki_enabled,
-                                     :snippets_enabled,
-                                     :container_registry_enabled,
-                                     :shared_runners_enabled,
+                                     :name,
                                      :namespace_id,
+                                     :only_allow_merge_if_build_succeeds,
+                                     :path,
                                      :public,
-                                     :visibility_level,
-                                     :import_url,
                                      :public_builds,
-                                     :only_allow_merge_if_build_succeeds,
-                                     :lfs_enabled]
+                                     :request_access_enabled,
+                                     :shared_runners_enabled,
+                                     :snippets_enabled,
+                                     :visibility_level,
+                                     :wiki_enabled]
         attrs = map_public_to_visibility_level(attrs)
         @project = ::Projects::CreateService.new(current_user, attrs).execute
         if @project.saved?
@@ -143,10 +145,10 @@ module API
       # Create new project for a specified user.  Only available to admin users.
       #
       # Parameters:
-      #   user_id (required) - The ID of a user
-      #   name (required) - name for new project
-      #   description (optional) - short project description
-      #   default_branch (optional) - 'master' by default
+      #   user_id (required)                - The ID of a user
+      #   name (required)                   - name for new project
+      #   description (optional)            - short project description
+      #   default_branch (optional)         - 'master' by default
       #   issues_enabled (optional)
       #   merge_requests_enabled (optional)
       #   builds_enabled (optional)
@@ -154,31 +156,33 @@ module API
       #   snippets_enabled (optional)
       #   container_registry_enabled (optional)
       #   shared_runners_enabled (optional)
-      #   public (optional) - if true same as setting visibility_level = 20
+      #   public (optional)                 - if true same as setting visibility_level = 20
       #   visibility_level (optional)
       #   import_url (optional)
       #   public_builds (optional)
       #   lfs_enabled (optional)
+      #   request_access_enabled (optional) - Allow users to request member access
       # Example Request
       #   POST /projects/user/:user_id
       post "user/:user_id" do
         authenticated_as_admin!
         user = User.find(params[:user_id])
-        attrs = attributes_for_keys [:name,
-                                     :description,
+        attrs = attributes_for_keys [:builds_enabled,
                                      :default_branch,
+                                     :description,
+                                     :import_url,
                                      :issues_enabled,
+                                     :lfs_enabled,
                                      :merge_requests_enabled,
-                                     :builds_enabled,
-                                     :wiki_enabled,
-                                     :snippets_enabled,
-                                     :shared_runners_enabled,
+                                     :name,
+                                     :only_allow_merge_if_build_succeeds,
                                      :public,
-                                     :visibility_level,
-                                     :import_url,
                                      :public_builds,
-                                     :only_allow_merge_if_build_succeeds,
-                                     :lfs_enabled]
+                                     :request_access_enabled,
+                                     :shared_runners_enabled,
+                                     :snippets_enabled,
+                                     :visibility_level,
+                                     :wiki_enabled]
         attrs = map_public_to_visibility_level(attrs)
         @project = ::Projects::CreateService.new(user, attrs).execute
         if @project.saved?
@@ -242,22 +246,23 @@ module API
       # Example Request
       #   PUT /projects/:id
       put ':id' do
-        attrs = attributes_for_keys [:name,
-                                     :path,
-                                     :description,
+        attrs = attributes_for_keys [:builds_enabled,
+                                     :container_registry_enabled,
                                      :default_branch,
+                                     :description,
                                      :issues_enabled,
+                                     :lfs_enabled,
                                      :merge_requests_enabled,
-                                     :builds_enabled,
-                                     :wiki_enabled,
-                                     :snippets_enabled,
-                                     :container_registry_enabled,
-                                     :shared_runners_enabled,
+                                     :name,
+                                     :only_allow_merge_if_build_succeeds,
+                                     :path,
                                      :public,
-                                     :visibility_level,
                                      :public_builds,
-                                     :only_allow_merge_if_build_succeeds,
-                                     :lfs_enabled]
+                                     :request_access_enabled,
+                                     :shared_runners_enabled,
+                                     :snippets_enabled,
+                                     :visibility_level,
+                                     :wiki_enabled]
         attrs = map_public_to_visibility_level(attrs)
         authorize_admin_project
         authorize! :rename_project, user_project if attrs[:name].present?

lib/ci/api/entities.rb

@@ -15,6 +15,15 @@ module Ci
         expose :filename, :size
       end
 
+      class BuildOptions < Grape::Entity
+        expose :image
+        expose :services
+        expose :artifacts
+        expose :cache
+        expose :dependencies
+        expose :after_script
+      end
+
       class Build < Grape::Entity
         expose :id, :ref, :tag, :sha, :status
         expose :name, :token, :stage

lib/ci/api/helpers.rb

@@ -14,12 +14,20 @@ module Ci
       end
 
       def authenticate_build_token!(build)
-        token = (params[BUILD_TOKEN_PARAM] || env[BUILD_TOKEN_HEADER]).to_s
-        forbidden! unless token && build.valid_token?(token)
+        forbidden! unless build_token_valid?(build)
       end
 
       def runner_registration_token_valid?
-        params[:token] == current_application_settings.runners_registration_token
+        ActiveSupport::SecurityUtils.variable_size_secure_compare(
+          params[:token],
+          current_application_settings.runners_registration_token)
+      end
+
+      def build_token_valid?(build)
+        token = (params[BUILD_TOKEN_PARAM] || env[BUILD_TOKEN_HEADER]).to_s
+
+        # We require to also check `runners_token` to maintain compatibility with old version of runners
+        token && (build.valid_token?(token) || build.project.valid_runners_token?(token))
       end
 
       def update_runner_last_contact(save: true)

lib/ci/gitlab_ci_yaml_processor.rb

@@ -60,7 +60,7 @@ module Ci
         name: job[:name].to_s,
         allow_failure: job[:allow_failure] || false,
         when: job[:when] || 'on_success',
-        environment: job[:environment],
+        environment: job[:environment_name],
         yaml_variables: yaml_variables(name),
         options: {
           image: job[:image],
@@ -69,6 +69,7 @@ module Ci
           cache: job[:cache],
           dependencies: job[:dependencies],
           after_script: job[:after_script],
+          environment: job[:environment],
         }.compact
       }
     end

lib/ci/mask_secret.rb

+module Ci::MaskSecret
+  class << self
+    def mask(value, token)
+      return value unless value.present? && token.present?
+
+      value.gsub(token, 'x' * token.length)
+    end
+  end
+end

lib/expand_variables.rb

+module ExpandVariables
+  class << self
+    def expand(value, variables)
+      # Convert hash array to variables
+      if variables.is_a?(Array)
+        variables = variables.reduce({}) do |hash, variable|
+          hash[variable[:key]] = variable[:value]
+          hash
+        end
+      end
+
+      value.gsub(/\$([a-zA-Z_][a-zA-Z0-9_]*)|\${\g<1>}|%\g<1>%/) do
+        variables[$1 || $2]
+      end
+    end
+  end
+end

lib/gitlab/auth.rb

 module Gitlab
   module Auth
-    Result = Struct.new(:user, :type)
+    class MissingPersonalTokenError < StandardError; end
 
     class << self
       def find_for_git_client(login, password, project:, ip:)
         raise "Must provide an IP for rate limiting" if ip.nil?
 
-        result = Result.new
+        result =
+          service_request_check(login, password, project) ||
+          build_access_token_check(login, password) ||
+          user_with_password_for_git(login, password) ||
+          oauth_access_token_check(login, password) ||
+          personal_access_token_check(login, password) ||
+          Gitlab::Auth::Result.new
 
-        if valid_ci_request?(login, password, project)
-          result.type = :ci
-        else
-          result = populate_result(login, password)
-        end
+        rate_limit!(ip, success: result.success?, login: login)
 
-        success = result.user.present? || [:ci, :missing_personal_token].include?(result.type)
-        rate_limit!(ip, success: success, login: login)
         result
       end
 
@@ -57,44 +57,31 @@ module Gitlab
 
       private
 
-      def valid_ci_request?(login, password, project)
+      def service_request_check(login, password, project)
         matched_login = /(?<service>^[a-zA-Z]*-ci)-token$/.match(login)
 
-        return false unless project && matched_login.present?
+        return unless project && matched_login.present?
 
         underscored_service = matched_login['service'].underscore
 
-        if underscored_service == 'gitlab_ci'
-          project && project.valid_build_token?(password)
-        elsif Service.available_services_names.include?(underscored_service)
+        if Service.available_services_names.include?(underscored_service)
           # We treat underscored_service as a trusted input because it is included
           # in the Service.available_services_names whitelist.
           service = project.public_send("#{underscored_service}_service")
 
-          service && service.activated? && service.valid_token?(password)
-        end
-      end
-
-      def populate_result(login, password)
-        result =
-          user_with_password_for_git(login, password) ||
-          oauth_access_token_check(login, password) ||
-          personal_access_token_check(login, password)
-
-        if result
-          result.type = nil unless result.user
-
-          if result.user && result.user.two_factor_enabled? && result.type == :gitlab_or_ldap
-            result.type = :missing_personal_token
+          if service && service.activated? && service.valid_token?(password)
+            Gitlab::Auth::Result.new(nil, project, :ci, build_authentication_abilities)
           end
         end
-
-        result || Result.new
       end
 
       def user_with_password_for_git(login, password)
         user = find_with_user_password(login, password)
-        Result.new(user, :gitlab_or_ldap) if user
+        return unless user
+
+        raise Gitlab::Auth::MissingPersonalTokenError if user.two_factor_enabled?
+
+        Gitlab::Auth::Result.new(user, nil, :gitlab_or_ldap, full_authentication_abilities)
       end
 
       def oauth_access_token_check(login, password)
@@ -102,7 +89,7 @@ module Gitlab
           token = Doorkeeper::AccessToken.by_token(password)
           if token && token.accessible?
             user = User.find_by(id: token.resource_owner_id)
-            Result.new(user, :oauth)
+            Gitlab::Auth::Result.new(user, nil, :oauth, read_authentication_abilities)
           end
         end
       end
@@ -111,9 +98,52 @@ module Gitlab
         if login && password
           user = User.find_by_personal_access_token(password)
           validation = User.by_login(login)
-          Result.new(user, :personal_token) if user == validation
+          Gitlab::Auth::Result.new(user, nil, :personal_token, full_authentication_abilities) if user.present? && user == validation
+        end
+      end
+
+      def build_access_token_check(login, password)
+        return unless login == 'gitlab-ci-token'
+        return unless password
+
+        build = ::Ci::Build.running.find_by_token(password)
+        return unless build
+        return unless build.project.builds_enabled?
+
+        if build.user
+          # If user is assigned to build, use restricted credentials of user
+          Gitlab::Auth::Result.new(build.user, build.project, :build, build_authentication_abilities)
+        else
+          # Otherwise use generic CI credentials (backward compatibility)
+          Gitlab::Auth::Result.new(nil, build.project, :ci, build_authentication_abilities)
         end
       end
+
+      public
+
+      def build_authentication_abilities
+        [
+          :read_project,
+          :build_download_code,
+          :build_read_container_image,
+          :build_create_container_image
+        ]
+      end
+
+      def read_authentication_abilities
+        [
+          :read_project,
+          :download_code,
+          :read_container_image
+        ]
+      end
+
+      def full_authentication_abilities
+        read_authentication_abilities + [
+          :push_code,
+          :create_container_image
+        ]
+      end
     end
   end
 end

lib/gitlab/auth/result.rb

+module Gitlab
+  module Auth
+    Result = Struct.new(:actor, :project, :type, :authentication_abilities) do
+      def ci?
+        type == :ci
+      end
+
+      def success?
+        actor.present? || type == :ci
+      end
+    end
+  end
+end

lib/gitlab/backend/shell.rb

@@ -6,7 +6,12 @@ module Gitlab
 
     KeyAdder = Struct.new(:io) do
       def add_key(id, key)
-        key.gsub!(/[[:space:]]+/, ' ').strip!
+        key = Gitlab::Shell.strip_key(key)
+        # Newline and tab are part of the 'protocol' used to transmit id+key to the other end
+        if key.include?("\t") || key.include?("\n")
+          raise Error.new("Invalid key: #{key.inspect}")
+        end
+
         io.puts("#{id}\t#{key}")
       end
     end
@@ -16,6 +21,10 @@ module Gitlab
         @version_required ||= File.read(Rails.root.
                                         join('GITLAB_SHELL_VERSION')).strip
       end
+
+      def strip_key(key)
+        key.split(/ /)[0, 2].join(' ')
+      end
     end
 
     # Init new repository
@@ -107,7 +116,7 @@ module Gitlab
     #
     def add_key(key_id, key_content)
       Gitlab::Utils.system_silent([gitlab_shell_keys_path,
-                                   'add-key', key_id, key_content])
+                                   'add-key', key_id, self.class.strip_key(key_content)])
     end
 
     # Batch-add keys to authorized_keys

lib/gitlab/ci/config/node/environment.rb

+module Gitlab
+  module Ci
+    class Config
+      module Node
+        ##
+        # Entry that represents an environment.
+        #
+        class Environment < Entry
+          include Validatable
+
+          ALLOWED_KEYS = %i[name url]
+
+          validations do
+            validate do
+              unless hash? || string?
+                errors.add(:config, 'should be a hash or a string')
+              end
+            end
+
+            validates :name, presence: true
+            validates :name,
+              type: {
+                with: String,
+                message: Gitlab::Regex.environment_name_regex_message }
+
+            validates :name,
+              format: {
+                with: Gitlab::Regex.environment_name_regex,
+                message: Gitlab::Regex.environment_name_regex_message }
+
+            with_options if: :hash? do
+              validates :config, allowed_keys: ALLOWED_KEYS
+
+              validates :url,
+                        length: { maximum: 255 },
+                        addressable_url: true,
+                        allow_nil: true
+            end
+          end
+
+          def hash?
+            @config.is_a?(Hash)
+          end
+
+          def string?
+            @config.is_a?(String)
+          end
+
+          def name
+            value[:name]
+          end
+
+          def url
+            value[:url]
+          end
+
+          def value
+            case @config
+            when String then { name: @config }
+            when Hash then @config
+            else {}
+            end
+          end
+        end
+      end
+    end
+  end
+end

lib/gitlab/ci/config/node/job.rb

@@ -13,7 +13,7 @@ module Gitlab
                             type stage when artifacts cache dependencies before_script
                             after_script variables environment]
 
-          attributes :tags, :allow_failure, :when, :environment, :dependencies
+          attributes :tags, :allow_failure, :when, :dependencies
 
           validations do
             validates :config, allowed_keys: ALLOWED_KEYS
@@ -29,58 +29,53 @@ module Gitlab
                 inclusion: { in: %w[on_success on_failure always manual],
                              message: 'should be on_success, on_failure, ' \
                                       'always or manual' }
-              validates :environment,
-                type: {
-                  with: String,
-                  message: Gitlab::Regex.environment_name_regex_message }
-              validates :environment,
-                format: {
-                  with: Gitlab::Regex.environment_name_regex,
-                  message: Gitlab::Regex.environment_name_regex_message }
 
               validates :dependencies, array_of_strings: true
             end
           end
 
-          node :before_script, Script,
+          node :before_script, Node::Script,
             description: 'Global before script overridden in this job.'
 
-          node :script, Commands,
+          node :script, Node::Commands,
             description: 'Commands that will be executed in this job.'
 
-          node :stage, Stage,
+          node :stage, Node::Stage,
             description: 'Pipeline stage this job will be executed into.'
 
-          node :type, Stage,
+          node :type, Node::Stage,
             description: 'Deprecated: stage this job will be executed into.'
 
-          node :after_script, Script,
+          node :after_script, Node::Script,
             description: 'Commands that will be executed when finishing job.'
 
-          node :cache, Cache,
+          node :cache, Node::Cache,
             description: 'Cache definition for this job.'
 
-          node :image, Image,
+          node :image, Node::Image,
             description: 'Image that will be used to execute this job.'
 
-          node :services, Services,
+          node :services, Node::Services,
             description: 'Services that will be used to execute this job.'
 
-          node :only, Trigger,
+          node :only, Node::Trigger,
             description: 'Refs policy this job will be executed for.'
 
-          node :except, Trigger,
+          node :except, Node::Trigger,
             description: 'Refs policy this job will be executed for.'
 
-          node :variables, Variables,
+          node :variables, Node::Variables,
             description: 'Environment variables available for this job.'
 
-          node :artifacts, Artifacts,
+          node :artifacts, Node::Artifacts,
             description: 'Artifacts configuration for this job.'
 
+          node :environment, Node::Environment,
+               description: 'Environment configuration for this job.'
+
           helpers :before_script, :script, :stage, :type, :after_script,
                   :cache, :image, :services, :only, :except, :variables,
-                  :artifacts, :commands
+                  :artifacts, :commands, :environment
 
           def compose!(deps = nil)
             super do
@@ -133,6 +128,8 @@ module Gitlab
               only: only,
               except: except,
               variables: variables_defined? ? variables : nil,
+              environment: environment_defined? ? environment : nil,
+              environment_name: environment_defined? ? environment[:name] : nil,
               artifacts: artifacts,
               after_script: after_script }
           end

lib/gitlab/git_access.rb

@@ -5,12 +5,13 @@ module Gitlab
     DOWNLOAD_COMMANDS = %w{ git-upload-pack git-upload-archive }
     PUSH_COMMANDS = %w{ git-receive-pack }
 
-    attr_reader :actor, :project, :protocol, :user_access
+    attr_reader :actor, :project, :protocol, :user_access, :authentication_abilities
 
-    def initialize(actor, project, protocol)
+    def initialize(actor, project, protocol, authentication_abilities:)
       @actor    = actor
       @project  = project
       @protocol = protocol
+      @authentication_abilities = authentication_abilities
       @user_access = UserAccess.new(user, project: project)
     end
 
@@ -60,14 +61,26 @@ module Gitlab
     end
 
     def user_download_access_check
-      unless user_access.can_do_action?(:download_code)
+      unless user_can_download_code? || build_can_download_code?
         return build_status_object(false, "You are not allowed to download code from this project.")
       end
 
       build_status_object(true)
     end
 
+    def user_can_download_code?
+      authentication_abilities.include?(:download_code) && user_access.can_do_action?(:download_code)
+    end
+
+    def build_can_download_code?
+      authentication_abilities.include?(:build_download_code) && user_access.can_do_action?(:build_download_code)
+    end
+
     def user_push_access_check(changes)
+      unless authentication_abilities.include?(:push_code)
+        return build_status_object(false, "You are not allowed to upload code for this project.")
+      end
+
       if changes.blank?
         return build_status_object(true)
       end

lib/gitlab/import_export.rb

@@ -2,7 +2,8 @@ module Gitlab
   module ImportExport
     extend self
 
-    VERSION = '0.1.3'
+    # For every version update, the version history in import_export.md has to be kept up to date.
+    VERSION = '0.1.4'
     FILENAME_LIMIT = 50
 
     def export_path(relative_path:)

lib/gitlab/import_export/import_export.yml

@@ -35,7 +35,9 @@ project_tree:
   - :deploy_keys
   - :services
   - :hooks
-  - :protected_branches
+  - protected_branches:
+    - :merge_access_levels
+    - :push_access_levels
   - :labels
   - milestones:
     - :events

lib/gitlab/import_export/relation_factory.rb

@@ -7,7 +7,9 @@ module Gitlab
                     variables: 'Ci::Variable',
                     triggers: 'Ci::Trigger',
                     builds: 'Ci::Build',
-                    hooks: 'ProjectHook' }.freeze
+                    hooks: 'ProjectHook',
+                    merge_access_levels: 'ProtectedBranch::MergeAccessLevel',
+                    push_access_levels: 'ProtectedBranch::PushAccessLevel' }.freeze
 
       USER_REFERENCES = %w[author_id assignee_id updated_by_id user_id].freeze
 
@@ -17,6 +19,8 @@ module Gitlab
 
       EXISTING_OBJECT_CHECK = %i[milestone milestones label labels].freeze
 
+      FINDER_ATTRIBUTES = %w[title project_id].freeze
+
       def self.create(*args)
         new(*args).create
       end
@@ -149,7 +153,7 @@ module Gitlab
       end
 
       def parsed_relation_hash
-        @relation_hash.reject { |k, _v| !relation_class.attribute_method?(k) }
+        @parsed_relation_hash ||= @relation_hash.reject { |k, _v| !relation_class.attribute_method?(k) }
       end
 
       def set_st_diffs
@@ -161,14 +165,30 @@ module Gitlab
         # Otherwise always create the record, skipping the extra SELECT clause.
         @existing_or_new_object ||= begin
           if EXISTING_OBJECT_CHECK.include?(@relation_name)
-            existing_object = relation_class.find_or_initialize_by(parsed_relation_hash.slice('title', 'project_id'))
-            existing_object.assign_attributes(parsed_relation_hash)
+            events = parsed_relation_hash.delete('events')
+
+            unless events.blank?
+              existing_object.assign_attributes(events: events)
+            end
+
             existing_object
           else
             relation_class.new(parsed_relation_hash)
           end
         end
       end
+
+      def existing_object
+        @existing_object ||=
+          begin
+            finder_hash = parsed_relation_hash.slice(*FINDER_ATTRIBUTES)
+            existing_object = relation_class.find_or_create_by(finder_hash)
+            # Done in two steps, as MySQL behaves differently than PostgreSQL using
+            # the +find_or_create_by+ method and does not return the ID the second time.
+            existing_object.update(parsed_relation_hash)
+            existing_object
+          end
+      end
     end
   end
 end

lib/gitlab/import_export/version_checker.rb

@@ -24,8 +24,8 @@ module Gitlab
       end
 
       def verify_version!(version)
-        if Gem::Version.new(version) > Gem::Version.new(Gitlab::ImportExport.version)
-          raise Gitlab::ImportExport::Error.new("Import version mismatch: Required <= #{Gitlab::ImportExport.version} but was #{version}")
+        if Gem::Version.new(version) != Gem::Version.new(Gitlab::ImportExport.version)
+          raise Gitlab::ImportExport::Error.new("Import version mismatch: Required #{Gitlab::ImportExport.version} but was #{version}")
         else
           true
         end

lib/gitlab/ldap/adapter.rb

@@ -70,7 +70,7 @@ module Gitlab
       private
 
       def user_options(field, value, limit)
-        options = { attributes: %W(#{config.uid} cn mail dn) }
+        options = { attributes: user_attributes }
         options[:size] = limit if limit
 
         if field.to_sym == :dn
@@ -98,6 +98,10 @@ module Gitlab
           filter
         end
       end
+
+      def user_attributes
+        %W(#{config.uid} cn mail dn)
+      end
     end
   end
 end

lib/gitlab/regex.rb

@@ -96,11 +96,11 @@ module Gitlab
     end
 
     def environment_name_regex
-      @environment_name_regex ||= /\A[a-zA-Z0-9_-]+\z/.freeze
+      @environment_name_regex ||= /\A[a-zA-Z0-9_\\\/\${}. -]+\z/.freeze
     end
 
     def environment_name_regex_message
-      "can contain only letters, digits, '-' and '_'."
+      "can contain only letters, digits, '-', '_', '/', '$', '{', '}', '.' and spaces"
     end
   end
 end

scripts/lint-doc.sh

@@ -10,6 +10,15 @@ then
   exit 1
 fi
 
+# Ensure that the CHANGELOG does not contain duplicate versions
+DUPLICATE_CHANGELOG_VERSIONS=$(grep --extended-regexp '^v [0-9.]+' CHANGELOG | sed 's| (unreleased)||' | sort | uniq -d)
+if [ "${DUPLICATE_CHANGELOG_VERSIONS}" != "" ]
+then
+  echo '✖ ERROR: Duplicate versions in CHANGELOG:' >&2
+  echo "${DUPLICATE_CHANGELOG_VERSIONS}" >&2
+  exit 1
+fi
+
 echo "✔ Linting passed"
 exit 0
 

spec/features/environments_spec.rb

@@ -150,7 +150,7 @@ feature 'Environments', feature: true do
 
       context 'for invalid name' do
         before do
-          fill_in('Name', with: 'name with spaces')
+          fill_in('Name', with: 'name,with,commas')
           click_on 'Save'
         end
 

spec/features/issues/filter_issues_spec.rb

@@ -101,7 +101,7 @@ describe 'Filter issues', feature: true do
       expect(find('.js-label-select .dropdown-toggle-text')).to have_content('No Label')
     end
 
-    it 'filters by no label' do
+    it 'filters by a label' do
       find('.dropdown-menu-labels a', text: label.title).click
       page.within '.labels-filter' do
         expect(page).to have_content label.title
@@ -109,7 +109,7 @@ describe 'Filter issues', feature: true do
       expect(find('.js-label-select .dropdown-toggle-text')).to have_content(label.title)
     end
 
-    it 'filters by wont fix labels' do
+    it "filters by `won't fix` and another label" do
       find('.dropdown-menu-labels a', text: label.title).click
       page.within '.labels-filter' do
         expect(page).to have_content wontfix.title
@@ -117,6 +117,33 @@ describe 'Filter issues', feature: true do
       end
       expect(find('.js-label-select .dropdown-toggle-text')).to have_content(wontfix.title)
     end
+
+    it "filters by `won't fix` label followed by another label after page load" do
+      find('.dropdown-menu-labels a', text: wontfix.title).click
+      # Close label dropdown to load
+      find('body').click
+      expect(find('.filtered-labels')).to have_content(wontfix.title)
+
+      find('.js-label-select').click
+      wait_for_ajax
+      find('.dropdown-menu-labels a', text: label.title).click
+      # Close label dropdown to load
+      find('body').click
+      expect(find('.filtered-labels')).to have_content(label.title)
+
+      find('.js-label-select').click
+      wait_for_ajax
+      expect(find('.dropdown-menu-labels li', text: wontfix.title)).to have_css('.is-active')
+      expect(find('.dropdown-menu-labels li', text: label.title)).to have_css('.is-active')
+    end
+
+    it "selects and unselects `won't fix`" do
+      find('.dropdown-menu-labels a', text: wontfix.title).click
+      find('.dropdown-menu-labels a', text: wontfix.title).click
+      # Close label dropdown to load
+      find('body').click
+      expect(page).not_to have_css('.filtered-labels')
+    end
   end
 
   describe 'Filter issues for assignee and label from issues#index' do

spec/features/milestone_spec.rb

@@ -3,9 +3,8 @@ require 'rails_helper'
 feature 'Milestone', feature: true do
   include WaitForAjax
 
-  let(:project) { create(:project, :public) }
+  let(:project) { create(:empty_project, :public) }
   let(:user)   { create(:user) }
-  let(:milestone) { create(:milestone, project: project, title: 8.7) }
 
   before do
     project.team << [user, :master]
@@ -13,7 +12,7 @@ feature 'Milestone', feature: true do
   end
 
   feature 'Create a milestone' do
-    scenario 'shows an informative message for a new issue' do
+    scenario 'shows an informative message for a new milestone' do
       visit new_namespace_project_milestone_path(project.namespace, project)
       page.within '.milestone-form' do
         fill_in "milestone_title", with: '8.7'
@@ -26,10 +25,26 @@ feature 'Milestone', feature: true do
 
   feature 'Open a milestone with closed issues' do
     scenario 'shows an informative message' do
+      milestone = create(:milestone, project: project, title: 8.7)
+
       create(:issue, title: "Bugfix1", project: project, milestone: milestone, state: "closed")
       visit namespace_project_milestone_path(project.namespace, project, milestone)
 
       expect(find('.alert-success')).to have_content('All issues for this milestone are closed. You may close this milestone now.')
     end
   end
+
+  feature 'Open a milestone with an existing title' do
+    scenario 'displays validation message' do
+      milestone = create(:milestone, project: project, title: 8.7)
+
+      visit new_namespace_project_milestone_path(project.namespace, project)
+      page.within '.milestone-form' do
+        fill_in "milestone_title", with: milestone.title
+      end
+      find('input[name="commit"]').click
+
+      expect(find('.alert-danger')).to have_content('Title has already been taken')
+    end
+  end
 end

spec/lib/ci/gitlab_ci_yaml_processor_spec.rb

@@ -754,6 +754,20 @@ module Ci
         it 'does return production' do
           expect(builds.size).to eq(1)
           expect(builds.first[:environment]).to eq(environment)
+          expect(builds.first[:options]).to include(environment: { name: environment })
+        end
+      end
+
+      context 'when hash is specified' do
+        let(:environment) do
+          { name: 'production',
+            url: 'http://production.gitlab.com' }
+        end
+
+        it 'does return production and URL' do
+          expect(builds.size).to eq(1)
+          expect(builds.first[:environment]).to eq(environment[:name])
+          expect(builds.first[:options]).to include(environment: environment)
         end
       end
 
@@ -770,15 +784,16 @@ module Ci
         let(:environment) { 1 }
 
         it 'raises error' do
-          expect { builds }.to raise_error("jobs:deploy_to_production environment #{Gitlab::Regex.environment_name_regex_message}")
+          expect { builds }.to raise_error(
+            'jobs:deploy_to_production:environment config should be a hash or a string')
         end
       end
 
       context 'is not a valid string' do
-        let(:environment) { 'production staging' }
+        let(:environment) { 'production:staging' }
 
         it 'raises error' do
-          expect { builds }.to raise_error("jobs:deploy_to_production environment #{Gitlab::Regex.environment_name_regex_message}")
+          expect { builds }.to raise_error("jobs:deploy_to_production:environment name #{Gitlab::Regex.environment_name_regex_message}")
         end
       end
     end

spec/lib/ci/mask_secret_spec.rb

+require 'spec_helper'
+
+describe Ci::MaskSecret, lib: true do
+  subject { described_class }
+
+  describe '#mask' do
+    it 'masks exact number of characters' do
+      expect(subject.mask('token', 'oke')).to eq('txxxn')
+    end
+
+    it 'masks multiple occurrences' do
+      expect(subject.mask('token token token', 'oke')).to eq('txxxn txxxn txxxn')
+    end
+
+    it 'does not mask if not found' do
+      expect(subject.mask('token', 'not')).to eq('token')
+    end
+  end
+end

spec/lib/expand_variables_spec.rb

+require 'spec_helper'
+
+describe ExpandVariables do
+  describe '#expand' do
+    subject { described_class.expand(value, variables) }
+
+    tests = [
+      { value: 'key',
+        result: 'key',
+        variables: []
+      },
+      { value: 'key$variable',
+        result: 'key',
+        variables: []
+      },
+      { value: 'key$variable',
+        result: 'keyvalue',
+        variables: [
+          { key: 'variable', value: 'value' }
+        ]
+      },
+      { value: 'key${variable}',
+        result: 'keyvalue',
+        variables: [
+          { key: 'variable', value: 'value' }
+        ]
+      },
+      { value: 'key$variable$variable2',
+        result: 'keyvalueresult',
+        variables: [
+          { key: 'variable', value: 'value' },
+          { key: 'variable2', value: 'result' },
+        ]
+      },
+      { value: 'key${variable}${variable2}',
+        result: 'keyvalueresult',
+        variables: [
+          { key: 'variable', value: 'value' },
+          { key: 'variable2', value: 'result' }
+        ]
+      },
+      { value: 'key$variable2$variable',
+        result: 'keyresultvalue',
+        variables: [
+          { key: 'variable', value: 'value' },
+          { key: 'variable2', value: 'result' },
+        ]
+      },
+      { value: 'key${variable2}${variable}',
+        result: 'keyresultvalue',
+        variables: [
+          { key: 'variable', value: 'value' },
+          { key: 'variable2', value: 'result' }
+        ]
+      },
+      { value: 'review/$CI_BUILD_REF_NAME',
+        result: 'review/feature/add-review-apps',
+        variables: [
+          { key: 'CI_BUILD_REF_NAME', value: 'feature/add-review-apps' }
+        ]
+      },
+    ]
+
+    tests.each do |test|
+      context "#{test[:value]} resolves to #{test[:result]}" do
+        let(:value) { test[:value] }
+        let(:variables) { test[:variables] }
+
+        it { is_expected.to eq(test[:result]) }
+      end
+    end
+  end
+end

spec/lib/gitlab/auth_spec.rb

@@ -4,15 +4,53 @@ describe Gitlab::Auth, lib: true do
   let(:gl_auth) { described_class }
 
   describe 'find_for_git_client' do
-    it 'recognizes CI' do
-      token = '123'
+    context 'build token' do
+      subject { gl_auth.find_for_git_client('gitlab-ci-token', build.token, project: project, ip: 'ip') }
+
+      context 'for running build' do
+        let!(:build) { create(:ci_build, :running) }
+        let(:project) { build.project }
+
+        before do
+          expect(gl_auth).to receive(:rate_limit!).with('ip', success: true, login: 'gitlab-ci-token')
+        end
+
+        it 'recognises user-less build' do
+          expect(subject).to eq(Gitlab::Auth::Result.new(nil, build.project, :ci, build_authentication_abilities))
+        end
+
+        it 'recognises user token' do
+          build.update(user: create(:user))
+
+          expect(subject).to eq(Gitlab::Auth::Result.new(build.user, build.project, :build, build_authentication_abilities))
+        end
+      end
+
+      (HasStatus::AVAILABLE_STATUSES - ['running']).each do |build_status|
+        context "for #{build_status} build" do
+          let!(:build) { create(:ci_build, status: build_status) }
+          let(:project) { build.project }
+
+          before do
+            expect(gl_auth).to receive(:rate_limit!).with('ip', success: false, login: 'gitlab-ci-token')
+          end
+
+          it 'denies authentication' do
+            expect(subject).to eq(Gitlab::Auth::Result.new)
+          end
+        end
+      end
+    end
+
+    it 'recognizes other ci services' do
       project = create(:empty_project)
-      project.update_attributes(runners_token: token)
+      project.create_drone_ci_service(active: true)
+      project.drone_ci_service.update(token: 'token')
 
       ip = 'ip'
 
-      expect(gl_auth).to receive(:rate_limit!).with(ip, success: true, login: 'gitlab-ci-token')
-      expect(gl_auth.find_for_git_client('gitlab-ci-token', token, project: project, ip: ip)).to eq(Gitlab::Auth::Result.new(nil, :ci))
+      expect(gl_auth).to receive(:rate_limit!).with(ip, success: true, login: 'drone-ci-token')
+      expect(gl_auth.find_for_git_client('drone-ci-token', 'token', project: project, ip: ip)).to eq(Gitlab::Auth::Result.new(nil, project, :ci, build_authentication_abilities))
     end
 
     it 'recognizes master passwords' do
@@ -20,7 +58,7 @@ describe Gitlab::Auth, lib: true do
       ip = 'ip'
 
       expect(gl_auth).to receive(:rate_limit!).with(ip, success: true, login: user.username)
-      expect(gl_auth.find_for_git_client(user.username, 'password', project: nil, ip: ip)).to eq(Gitlab::Auth::Result.new(user, :gitlab_or_ldap))
+      expect(gl_auth.find_for_git_client(user.username, 'password', project: nil, ip: ip)).to eq(Gitlab::Auth::Result.new(user, nil, :gitlab_or_ldap, full_authentication_abilities))
     end
 
     it 'recognizes OAuth tokens' do
@@ -30,7 +68,7 @@ describe Gitlab::Auth, lib: true do
       ip = 'ip'
 
       expect(gl_auth).to receive(:rate_limit!).with(ip, success: true, login: 'oauth2')
-      expect(gl_auth.find_for_git_client("oauth2", token.token, project: nil, ip: ip)).to eq(Gitlab::Auth::Result.new(user, :oauth))
+      expect(gl_auth.find_for_git_client("oauth2", token.token, project: nil, ip: ip)).to eq(Gitlab::Auth::Result.new(user, nil, :oauth, read_authentication_abilities))
     end
 
     it 'returns double nil for invalid credentials' do
@@ -92,4 +130,30 @@ describe Gitlab::Auth, lib: true do
       end
     end
   end
+
+  private
+
+  def build_authentication_abilities
+    [
+      :read_project,
+      :build_download_code,
+      :build_read_container_image,
+      :build_create_container_image
+    ]
+  end
+
+  def read_authentication_abilities
+    [
+      :read_project,
+      :download_code,
+      :read_container_image
+    ]
+  end
+
+  def full_authentication_abilities
+    read_authentication_abilities + [
+      :push_code,
+      :create_container_image
+    ]
+  end
 end

spec/lib/gitlab/backend/shell_spec.rb

 require 'spec_helper'
+require 'stringio'
 
 describe Gitlab::Shell, lib: true do
   let(:project) { double('Project', id: 7, path: 'diaspora') }
@@ -44,15 +45,38 @@ describe Gitlab::Shell, lib: true do
     end
   end
 
+  describe '#add_key' do
+    it 'removes trailing garbage' do
+      allow(gitlab_shell).to receive(:gitlab_shell_keys_path).and_return(:gitlab_shell_keys_path)
+      expect(Gitlab::Utils).to receive(:system_silent).with(
+        [:gitlab_shell_keys_path, 'add-key', 'key-123', 'ssh-rsa foobar']
+      )
+
+      gitlab_shell.add_key('key-123', 'ssh-rsa foobar trailing garbage')
+    end
+  end
+
   describe Gitlab::Shell::KeyAdder, lib: true do
     describe '#add_key' do
-      it 'normalizes space characters in the key' do
-        io = spy
+      it 'removes trailing garbage' do
+        io = spy(:io)
         adder = described_class.new(io)
 
-        adder.add_key('key-42', "sha-rsa foo\tbar\tbaz")
+        adder.add_key('key-42', "ssh-rsa foo bar\tbaz")
+
+        expect(io).to have_received(:puts).with("key-42\tssh-rsa foo")
+      end
+
+      it 'raises an exception if the key contains a tab' do
+        expect do
+          described_class.new(StringIO.new).add_key('key-42', "ssh-rsa\tfoobar")
+        end.to raise_error(Gitlab::Shell::Error)
+      end
 
-        expect(io).to have_received(:puts).with("key-42\tsha-rsa foo bar baz")
+      it 'raises an exception if the key contains a newline' do
+        expect do
+          described_class.new(StringIO.new).add_key('key-42', "ssh-rsa foobar\nssh-rsa pawned")
+        end.to raise_error(Gitlab::Shell::Error)
       end
     end
   end

spec/lib/gitlab/ci/config/node/environment_spec.rb

+require 'spec_helper'
+
+describe Gitlab::Ci::Config::Node::Environment do
+  let(:entry) { described_class.new(config) }
+
+  before { entry.compose! }
+
+  context 'when configuration is a string' do
+    let(:config) { 'production' }
+
+    describe '#string?' do
+      it 'is string configuration' do
+        expect(entry).to be_string
+      end
+    end
+
+    describe '#hash?' do
+      it 'is not hash configuration' do
+        expect(entry).not_to be_hash
+      end
+    end
+
+    describe '#valid?' do
+      it 'is valid' do
+        expect(entry).to be_valid
+      end
+    end
+
+    describe '#value' do
+      it 'returns valid hash' do
+        expect(entry.value).to eq(name: 'production')
+      end
+    end
+
+    describe '#name' do
+      it 'returns environment name' do
+        expect(entry.name).to eq 'production'
+      end
+    end
+
+    describe '#url' do
+      it 'returns environment url' do
+        expect(entry.url).to be_nil
+      end
+    end
+  end
+
+  context 'when configuration is a hash' do
+    let(:config) do
+      { name: 'development', url: 'https://example.gitlab.com' }
+    end
+
+    describe '#string?' do
+      it 'is not string configuration' do
+        expect(entry).not_to be_string
+      end
+    end
+
+    describe '#hash?' do
+      it 'is hash configuration' do
+        expect(entry).to be_hash
+      end
+    end
+
+    describe '#valid?' do
+      it 'is valid' do
+        expect(entry).to be_valid
+      end
+    end
+
+    describe '#value' do
+      it 'returns valid hash' do
+        expect(entry.value).to eq config
+      end
+    end
+
+    describe '#name' do
+      it 'returns environment name' do
+        expect(entry.name).to eq 'development'
+      end
+    end
+
+    describe '#url' do
+      it 'returns environment url' do
+        expect(entry.url).to eq 'https://example.gitlab.com'
+      end
+    end
+  end
+
+  context 'when variables are used for environment' do
+    let(:config) do
+      { name: 'review/$CI_BUILD_REF_NAME',
+        url: 'https://$CI_BUILD_REF_NAME.review.gitlab.com' }
+    end
+
+    describe '#valid?' do
+      it 'is valid' do
+        expect(entry).to be_valid
+      end
+    end
+  end
+
+  context 'when configuration is invalid' do
+    context 'when configuration is an array' do
+      let(:config) { ['env'] }
+
+      describe '#valid?' do
+        it 'is not valid' do
+          expect(entry).not_to be_valid
+        end
+      end
+
+      describe '#errors' do
+        it 'contains error about invalid type' do
+          expect(entry.errors)
+            .to include 'environment config should be a hash or a string'
+        end
+      end
+    end
+
+    context 'when environment name is not present' do
+      let(:config) { { url: 'https://example.gitlab.com' } }
+
+      describe '#valid?' do
+        it 'is not valid' do
+          expect(entry).not_to be_valid
+        end
+      end
+
+      describe '#errors?' do
+        it 'contains error about missing environment name' do
+          expect(entry.errors)
+            .to include "environment name can't be blank"
+        end
+      end
+    end
+
+    context 'when invalid URL is used' do
+      let(:config) { { name: 'test', url: 'invalid-example.gitlab.com' } }
+
+      describe '#valid?' do
+        it 'is not valid' do
+          expect(entry).not_to be_valid
+        end
+      end
+
+      describe '#errors?' do
+        it 'contains error about invalid URL' do
+          expect(entry.errors)
+            .to include "environment url must be a valid url"
+        end
+      end
+    end
+  end
+end

spec/lib/gitlab/git_access_spec.rb

 require 'spec_helper'
 
 describe Gitlab::GitAccess, lib: true do
-  let(:access) { Gitlab::GitAccess.new(actor, project, 'web') }
+  let(:access) { Gitlab::GitAccess.new(actor, project, 'web', authentication_abilities: authentication_abilities) }
   let(:project) { create(:project) }
   let(:user) { create(:user) }
   let(:actor) { user }
+  let(:authentication_abilities) do
+    [
+      :read_project,
+      :download_code,
+      :push_code
+    ]
+  end
 
   describe '#check with single protocols allowed' do
     def disable_protocol(protocol)
@@ -15,7 +22,7 @@ describe Gitlab::GitAccess, lib: true do
     context 'ssh disabled' do
       before do
         disable_protocol('ssh')
-        @acc = Gitlab::GitAccess.new(actor, project, 'ssh')
+        @acc = Gitlab::GitAccess.new(actor, project, 'ssh', authentication_abilities: authentication_abilities)
       end
 
       it 'blocks ssh git push' do
@@ -30,7 +37,7 @@ describe Gitlab::GitAccess, lib: true do
     context 'http disabled' do
       before do
         disable_protocol('http')
-        @acc = Gitlab::GitAccess.new(actor, project, 'http')
+        @acc = Gitlab::GitAccess.new(actor, project, 'http', authentication_abilities: authentication_abilities)
       end
 
       it 'blocks http push' do
@@ -111,6 +118,36 @@ describe Gitlab::GitAccess, lib: true do
         end
       end
     end
+
+    describe 'build authentication_abilities permissions' do
+      let(:authentication_abilities) { build_authentication_abilities }
+
+      describe 'reporter user' do
+        before { project.team << [user, :reporter] }
+
+        context 'pull code' do
+          it { expect(subject).to be_allowed }
+        end
+      end
+
+      describe 'admin user' do
+        let(:user) { create(:admin) }
+
+        context 'when member of the project' do
+          before { project.team << [user, :reporter] }
+
+          context 'pull code' do
+            it { expect(subject).to be_allowed }
+          end
+        end
+
+        context 'when is not member of the project' do
+          context 'pull code' do
+            it { expect(subject).not_to be_allowed }
+          end
+        end
+      end
+    end
   end
 
   describe 'push_access_check' do
@@ -283,38 +320,71 @@ describe Gitlab::GitAccess, lib: true do
     end
   end
 
-  describe 'deploy key permissions' do
-    let(:key) { create(:deploy_key) }
-    let(:actor) { key }
+  shared_examples 'can not push code' do
+    subject { access.check('git-receive-pack', '_any') }
+
+    context 'when project is authorized' do
+      before { authorize }
 
-    context 'push code' do
-      subject { access.check('git-receive-pack', '_any') }
+      it { expect(subject).not_to be_allowed }
+    end
 
-      context 'when project is authorized' do
-        before { key.projects << project }
+    context 'when unauthorized' do
+      context 'to public project' do
+        let(:project) { create(:project, :public) }
 
         it { expect(subject).not_to be_allowed }
       end
 
-      context 'when unauthorized' do
-        context 'to public project' do
-          let(:project) { create(:project, :public) }
+      context 'to internal project' do
+        let(:project) { create(:project, :internal) }
 
-          it { expect(subject).not_to be_allowed }
-        end
+        it { expect(subject).not_to be_allowed }
+      end
 
-        context 'to internal project' do
-          let(:project) { create(:project, :internal) }
+      context 'to private project' do
+        let(:project) { create(:project, :internal) }
 
-          it { expect(subject).not_to be_allowed }
-        end
+        it { expect(subject).not_to be_allowed }
+      end
+    end
+  end
 
-        context 'to private project' do
-          let(:project) { create(:project, :internal) }
+  describe 'build authentication abilities' do
+    let(:authentication_abilities) { build_authentication_abilities }
 
-          it { expect(subject).not_to be_allowed }
-        end
+    it_behaves_like 'can not push code' do
+      def authorize
+        project.team << [user, :reporter]
       end
     end
   end
+
+  describe 'deploy key permissions' do
+    let(:key) { create(:deploy_key) }
+    let(:actor) { key }
+
+    it_behaves_like 'can not push code' do
+      def authorize
+        key.projects << project
+      end
+    end
+  end
+
+  private
+
+  def build_authentication_abilities
+    [
+      :read_project,
+      :build_download_code
+    ]
+  end
+
+  def full_authentication_abilities
+    [
+      :read_project,
+      :download_code,
+      :push_code
+    ]
+  end
 end

spec/lib/gitlab/git_access_wiki_spec.rb

 require 'spec_helper'
 
 describe Gitlab::GitAccessWiki, lib: true do
-  let(:access) { Gitlab::GitAccessWiki.new(user, project, 'web') }
+  let(:access) { Gitlab::GitAccessWiki.new(user, project, 'web', authentication_abilities: authentication_abilities) }
   let(:project) { create(:project) }
   let(:user) { create(:user) }
+  let(:authentication_abilities) do
+    [
+      :read_project,
+      :download_code,
+      :push_code
+    ]
+  end
 
   describe 'push_allowed?' do
     before do

spec/lib/gitlab/import_export/project.json

@@ -24,7 +24,7 @@
       "test_ee_field": "test",
       "milestone": {
         "id": 1,
-        "title": "v0.0",
+        "title": "test milestone",
         "project_id": 8,
         "description": "test milestone",
         "due_date": null,
@@ -51,7 +51,7 @@
         {
           "id": 2,
           "label_id": 2,
-          "target_id": 3,
+          "target_id": 40,
           "target_type": "Issue",
           "created_at": "2016-07-22T08:57:02.840Z",
           "updated_at": "2016-07-22T08:57:02.840Z",
@@ -281,6 +281,31 @@
       "deleted_at": null,
       "due_date": null,
       "moved_to_id": null,
+      "milestone": {
+        "id": 1,
+        "title": "test milestone",
+        "project_id": 8,
+        "description": "test milestone",
+        "due_date": null,
+        "created_at": "2016-06-14T15:02:04.415Z",
+        "updated_at": "2016-06-14T15:02:04.415Z",
+        "state": "active",
+        "iid": 1,
+        "events": [
+          {
+            "id": 487,
+            "target_type": "Milestone",
+            "target_id": 1,
+            "title": null,
+            "data": null,
+            "project_id": 46,
+            "created_at": "2016-06-14T15:02:04.418Z",
+            "updated_at": "2016-06-14T15:02:04.418Z",
+            "action": 1,
+            "author_id": 18
+          }
+        ]
+      },
       "notes": [
         {
           "id": 359,
@@ -494,6 +519,27 @@
       "deleted_at": null,
       "due_date": null,
       "moved_to_id": null,
+      "label_links": [
+        {
+          "id": 99,
+          "label_id": 2,
+          "target_id": 38,
+          "target_type": "Issue",
+          "created_at": "2016-07-22T08:57:02.840Z",
+          "updated_at": "2016-07-22T08:57:02.840Z",
+          "label": {
+            "id": 2,
+            "title": "test2",
+            "color": "#428bca",
+            "project_id": 8,
+            "created_at": "2016-07-22T08:55:44.161Z",
+            "updated_at": "2016-07-22T08:55:44.161Z",
+            "template": false,
+            "description": "",
+            "priority": null
+          }
+        }
+      ],
       "notes": [
         {
           "id": 367,
@@ -6478,7 +6524,7 @@
     {
       "id": 37,
       "project_id": 5,
-      "ref": "master",
+      "ref": null,
       "sha": "048721d90c449b244b7b4c53a9186b04330174ec",
       "before_sha": null,
       "push_data": null,
@@ -7301,6 +7347,30 @@
 
   ],
   "protected_branches": [
-
+    {
+      "id": 1,
+      "project_id": 9,
+      "name": "master",
+      "created_at": "2016-08-30T07:32:52.426Z",
+      "updated_at": "2016-08-30T07:32:52.426Z",
+      "merge_access_levels": [
+        {
+          "id": 1,
+          "protected_branch_id": 1,
+          "access_level": 40,
+          "created_at": "2016-08-30T07:32:52.458Z",
+          "updated_at": "2016-08-30T07:32:52.458Z"
+        }
+      ],
+      "push_access_levels": [
+        {
+          "id": 1,
+          "protected_branch_id": 1,
+          "access_level": 40,
+          "created_at": "2016-08-30T07:32:52.490Z",
+          "updated_at": "2016-08-30T07:32:52.490Z"
+        }
+      ]
+    }
   ]
-}
+}
\ No newline at end of file

spec/lib/gitlab/import_export/project_tree_restorer_spec.rb

@@ -29,12 +29,30 @@ describe Gitlab::ImportExport::ProjectTreeRestorer, services: true do
         expect(project.project_feature.merge_requests_access_level).to eq(ProjectFeature::ENABLED)
       end
 
+      it 'has the same label associated to two issues' do
+        restored_project_json
+
+        expect(Label.first.issues.count).to eq(2)
+      end
+
+      it 'has milestones associated to two separate issues' do
+        restored_project_json
+
+        expect(Milestone.find_by_description('test milestone').issues.count).to eq(2)
+      end
+
       it 'creates a valid pipeline note' do
         restored_project_json
 
         expect(Ci::Pipeline.first.notes).not_to be_empty
       end
 
+      it 'restores pipelines with missing ref' do
+        restored_project_json
+
+        expect(Ci::Pipeline.where(ref: nil)).not_to be_empty
+      end
+
       it 'restores the correct event with symbolised data' do
         restored_project_json
 
@@ -49,6 +67,18 @@ describe Gitlab::ImportExport::ProjectTreeRestorer, services: true do
         expect(issue.reload.updated_at.to_s).to eq('2016-06-14 15:02:47 UTC')
       end
 
+      it 'contains the merge access levels on a protected branch' do
+        restored_project_json
+
+        expect(ProtectedBranch.first.merge_access_levels).not_to be_empty
+      end
+
+      it 'contains the push access levels on a protected branch' do
+        restored_project_json
+
+        expect(ProtectedBranch.first.push_access_levels).not_to be_empty
+      end
+
       context 'event at forth level of the tree' do
         let(:event) { Event.where(title: 'test levels').first }
 
@@ -77,12 +107,6 @@ describe Gitlab::ImportExport::ProjectTreeRestorer, services: true do
         expect(Label.first.label_links.first.target).not_to be_nil
       end
 
-      it 'has milestones associated to issues' do
-        restored_project_json
-
-        expect(Milestone.find_by_description('test milestone').issues).not_to be_empty
-      end
-
       context 'Merge requests' do
         before do
           restored_project_json

spec/lib/gitlab/import_export/version_checker_spec.rb

@@ -23,7 +23,7 @@ describe Gitlab::ImportExport::VersionChecker, services: true do
       it 'shows the correct error message' do
         described_class.check!(shared: shared)
 
-        expect(shared.errors.first).to eq("Import version mismatch: Required <= #{Gitlab::ImportExport.version} but was #{version}")
+        expect(shared.errors.first).to eq("Import version mismatch: Required #{Gitlab::ImportExport.version} but was #{version}")
       end
     end
   end

spec/models/build_spec.rb

@@ -88,9 +88,7 @@ describe Ci::Build, models: true do
   end
 
   describe '#trace' do
-    subject { build.trace_html }
-
-    it { is_expected.to be_empty }
+    it { expect(build.trace).to be_nil }
 
     context 'when build.trace contains text' do
       let(:text) { 'example output' }
@@ -98,16 +96,80 @@ describe Ci::Build, models: true do
         build.trace = text
       end
 
-      it { is_expected.to include(text) }
-      it { expect(subject.length).to be >= text.length }
+      it { expect(build.trace).to eq(text) }
+    end
+
+    context 'when build.trace hides runners token' do
+      let(:token) { 'my_secret_token' }
+
+      before do
+        build.update(trace: token)
+        build.project.update(runners_token: token)
+      end
+
+      it { expect(build.trace).not_to include(token) }
+      it { expect(build.raw_trace).to include(token) }
+    end
+
+    context 'when build.trace hides build token' do
+      let(:token) { 'my_secret_token' }
+
+      before do
+        build.update(trace: token)
+        build.update(token: token)
+      end
+
+      it { expect(build.trace).not_to include(token) }
+      it { expect(build.raw_trace).to include(token) }
+    end
+  end
+
+  describe '#raw_trace' do
+    subject { build.raw_trace }
+
+    context 'when build.trace hides runners token' do
+      let(:token) { 'my_secret_token' }
+
+      before do
+        build.project.update(runners_token: token)
+        build.update(trace: token)
+      end
+
+      it { is_expected.not_to include(token) }
+    end
+
+    context 'when build.trace hides build token' do
+      let(:token) { 'my_secret_token' }
+
+      before do
+        build.update(token: token)
+        build.update(trace: token)
+      end
+
+      it { is_expected.not_to include(token) }
+    end
+  end
+
+  context '#append_trace' do
+    subject { build.trace_html }
+
+    context 'when build.trace hides runners token' do
+      let(:token) { 'my_secret_token' }
+
+      before do
+        build.project.update(runners_token: token)
+        build.append_trace(token, 0)
+      end
+
+      it { is_expected.not_to include(token) }
     end
 
-    context 'when build.trace hides token' do
+    context 'when build.trace hides build token' do
       let(:token) { 'my_secret_token' }
 
       before do
-        build.project.update_attributes(runners_token: token)
-        build.update_attributes(trace: token)
+        build.update(token: token)
+        build.append_trace(token, 0)
       end
 
       it { is_expected.not_to include(token) }

spec/models/ci/build_spec.rb

@@ -8,7 +8,7 @@ describe Ci::Build, models: true do
     it 'obfuscates project runners token' do
       allow(build).to receive(:raw_trace).and_return("Test: #{build.project.runners_token}")
 
-      expect(build.trace).to eq("Test: xxxxxx")
+      expect(build.trace).to eq("Test: xxxxxxxxxxxxxxxxxxxx")
     end
 
     it 'empty project runners token' do

spec/models/environment_spec.rb

@@ -63,4 +63,20 @@ describe Environment, models: true do
       end
     end
   end
+
+  describe '#environment_type' do
+    subject { environment.environment_type }
+
+    it 'sets a environment type if name has multiple segments' do
+      environment.update!(name: 'production/worker.gitlab.com')
+
+      is_expected.to eq('production')
+    end
+
+    it 'nullifies a type if it\'s a simple name' do
+      environment.update!(name: 'production')
+
+      is_expected.to be_nil
+    end
+  end
 end

spec/models/event_spec.rb

@@ -16,18 +16,12 @@ describe Event, models: true do
 
   describe 'Callbacks' do
     describe 'after_create :reset_project_activity' do
-      let(:project) { create(:project) }
+      let(:project) { create(:empty_project) }
 
-      context "project's last activity was less than 5 minutes ago" do
-        it 'does not update project.last_activity_at if it has been touched less than 5 minutes ago' do
-          create_event(project, project.owner)
-          project.update_column(:last_activity_at, 5.minutes.ago)
-          project_last_activity_at = project.last_activity_at
+      it 'calls the reset_project_activity method' do
+        expect_any_instance_of(Event).to receive(:reset_project_activity)
 
-          create_event(project, project.owner)
-
-          expect(project.last_activity_at).to eq(project_last_activity_at)
-        end
+        create_event(project, project.owner)
       end
     end
   end
@@ -161,6 +155,35 @@ describe Event, models: true do
     end
   end
 
+  describe '#reset_project_activity' do
+    let(:project) { create(:empty_project) }
+
+    context 'when a project was updated less than 1 hour ago' do
+      it 'does not update the project' do
+        project.update(last_activity_at: Time.now)
+
+        expect(project).not_to receive(:update_column).
+          with(:last_activity_at, a_kind_of(Time))
+
+        create_event(project, project.owner)
+      end
+    end
+
+    context 'when a project was updated more than 1 hour ago' do
+      it 'updates the project' do
+        project.update(last_activity_at: 1.year.ago)
+
+        expect_any_instance_of(Gitlab::ExclusiveLease).
+          to receive(:try_obtain).and_return(true)
+
+        expect(project).to receive(:update_column).
+          with(:last_activity_at, a_kind_of(Time))
+
+        create_event(project, project.owner)
+      end
+    end
+  end
+
   def create_event(project, user, attrs = {})
     data = {
       before: Gitlab::Git::BLANK_SHA,

spec/requests/api/groups_spec.rb

@@ -120,10 +120,11 @@ describe API::API, api: true  do
 
     context 'when authenticated as the group owner' do
       it 'updates the group' do
-        put api("/groups/#{group1.id}", user1), name: new_group_name
+        put api("/groups/#{group1.id}", user1), name: new_group_name, request_access_enabled: true
 
         expect(response).to have_http_status(200)
         expect(json_response['name']).to eq(new_group_name)
+        expect(json_response['request_access_enabled']).to eq(true)
       end
 
       it 'returns 404 for a non existing group' do
@@ -238,8 +239,14 @@ describe API::API, api: true  do
 
     context "when authenticated as user with group permissions" do
       it "creates group" do
-        post api("/groups", user3), attributes_for(:group)
+        group = attributes_for(:group, { request_access_enabled: false })
+
+        post api("/groups", user3), group
         expect(response).to have_http_status(201)
+
+        expect(json_response["name"]).to eq(group[:name])
+        expect(json_response["path"]).to eq(group[:path])
+        expect(json_response["request_access_enabled"]).to eq(group[:request_access_enabled])
       end
 
       it "does not create group, duplicate" do

spec/requests/api/members_spec.rb

@@ -30,20 +30,29 @@ describe API::Members, api: true  do
         let(:route) { get api("/#{source_type.pluralize}/#{source.id}/members", stranger) }
       end
 
-      context 'when authenticated as a non-member' do
-        %i[access_requester stranger].each do |type|
-          context "as a #{type}" do
-            it 'returns 200' do
-              user = public_send(type)
-              get api("/#{source_type.pluralize}/#{source.id}/members", user)
+      %i[master developer access_requester stranger].each do |type|
+        context "when authenticated as a #{type}" do
+          it 'returns 200' do
+            user = public_send(type)
+            get api("/#{source_type.pluralize}/#{source.id}/members", user)
 
-              expect(response).to have_http_status(200)
-              expect(json_response.size).to eq(2)
-            end
+            expect(response).to have_http_status(200)
+            expect(json_response.size).to eq(2)
+            expect(json_response.map { |u| u['id'] }).to match_array [master.id, developer.id]
           end
         end
       end
 
+      it 'does not return invitees' do
+        create(:"#{source_type}_member", invite_token: '123', invite_email: 'test@abc.com', source: source, user: nil)
+
+        get api("/#{source_type.pluralize}/#{source.id}/members", developer)
+
+        expect(response).to have_http_status(200)
+        expect(json_response.size).to eq(2)
+        expect(json_response.map { |u| u['id'] }).to match_array [master.id, developer.id]
+      end
+
       it 'finds members with query string' do
         get api("/#{source_type.pluralize}/#{source.id}/members", developer), query: master.username
 

spec/requests/api/milestones_spec.rb

@@ -3,7 +3,7 @@ require 'spec_helper'
 describe API::API, api: true  do
   include ApiHelpers
   let(:user) { create(:user) }
-  let!(:project) { create(:project, namespace: user.namespace ) }
+  let!(:project) { create(:empty_project, namespace: user.namespace ) }
   let!(:closed_milestone) { create(:closed_milestone, project: project) }
   let!(:milestone) { create(:milestone, project: project) }
 
@@ -12,6 +12,7 @@ describe API::API, api: true  do
   describe 'GET /projects/:id/milestones' do
     it 'returns project milestones' do
       get api("/projects/#{project.id}/milestones", user)
+
       expect(response).to have_http_status(200)
       expect(json_response).to be_an Array
       expect(json_response.first['title']).to eq(milestone.title)
@@ -19,6 +20,7 @@ describe API::API, api: true  do
 
     it 'returns a 401 error if user not authenticated' do
       get api("/projects/#{project.id}/milestones")
+
       expect(response).to have_http_status(401)
     end
 
@@ -44,6 +46,7 @@ describe API::API, api: true  do
   describe 'GET /projects/:id/milestones/:milestone_id' do
     it 'returns a project milestone by id' do
       get api("/projects/#{project.id}/milestones/#{milestone.id}", user)
+
       expect(response).to have_http_status(200)
       expect(json_response['title']).to eq(milestone.title)
       expect(json_response['iid']).to eq(milestone.iid)
@@ -60,11 +63,13 @@ describe API::API, api: true  do
 
     it 'returns 401 error if user not authenticated' do
       get api("/projects/#{project.id}/milestones/#{milestone.id}")
+
       expect(response).to have_http_status(401)
     end
 
     it 'returns a 404 error if milestone id not found' do
       get api("/projects/#{project.id}/milestones/1234", user)
+
       expect(response).to have_http_status(404)
     end
   end
@@ -72,6 +77,7 @@ describe API::API, api: true  do
   describe 'POST /projects/:id/milestones' do
     it 'creates a new project milestone' do
       post api("/projects/#{project.id}/milestones", user), title: 'new milestone'
+
       expect(response).to have_http_status(201)
       expect(json_response['title']).to eq('new milestone')
       expect(json_response['description']).to be_nil
@@ -80,6 +86,7 @@ describe API::API, api: true  do
     it 'creates a new project milestone with description and due date' do
       post api("/projects/#{project.id}/milestones", user),
         title: 'new milestone', description: 'release', due_date: '2013-03-02'
+
       expect(response).to have_http_status(201)
       expect(json_response['description']).to eq('release')
       expect(json_response['due_date']).to eq('2013-03-02')
@@ -87,6 +94,14 @@ describe API::API, api: true  do
 
     it 'returns a 400 error if title is missing' do
       post api("/projects/#{project.id}/milestones", user)
+
+      expect(response).to have_http_status(400)
+    end
+
+    it 'returns a 400 error if params are invalid (duplicate title)' do
+      post api("/projects/#{project.id}/milestones", user),
+        title: milestone.title, description: 'release', due_date: '2013-03-02'
+
       expect(response).to have_http_status(400)
     end
   end
@@ -95,6 +110,7 @@ describe API::API, api: true  do
     it 'updates a project milestone' do
       put api("/projects/#{project.id}/milestones/#{milestone.id}", user),
         title: 'updated title'
+
       expect(response).to have_http_status(200)
       expect(json_response['title']).to eq('updated title')
     end
@@ -102,6 +118,7 @@ describe API::API, api: true  do
     it 'returns a 404 error if milestone id not found' do
       put api("/projects/#{project.id}/milestones/1234", user),
         title: 'updated title'
+
       expect(response).to have_http_status(404)
     end
   end
@@ -131,6 +148,7 @@ describe API::API, api: true  do
     end
     it 'returns project issues for a particular milestone' do
       get api("/projects/#{project.id}/milestones/#{milestone.id}/issues", user)
+
       expect(response).to have_http_status(200)
       expect(json_response).to be_an Array
       expect(json_response.first['milestone']['title']).to eq(milestone.title)
@@ -138,11 +156,12 @@ describe API::API, api: true  do
 
     it 'returns a 401 error if user not authenticated' do
       get api("/projects/#{project.id}/milestones/#{milestone.id}/issues")
+
       expect(response).to have_http_status(401)
     end
 
     describe 'confidential issues' do
-      let(:public_project) { create(:project, :public) }
+      let(:public_project) { create(:empty_project, :public) }
       let(:milestone) { create(:milestone, project: public_project) }
       let(:issue) { create(:issue, project: public_project) }
       let(:confidential_issue) { create(:issue, confidential: true, project: public_project) }

spec/requests/api/projects_spec.rb

@@ -225,7 +225,8 @@ describe API::API, api: true  do
         issues_enabled: false,
         merge_requests_enabled: false,
         wiki_enabled: false,
-        only_allow_merge_if_build_succeeds: false
+        only_allow_merge_if_build_succeeds: false,
+        request_access_enabled: true
       })
 
       post api('/projects', user), project
@@ -352,7 +353,8 @@ describe API::API, api: true  do
         description: FFaker::Lorem.sentence,
         issues_enabled: false,
         merge_requests_enabled: false,
-        wiki_enabled: false
+        wiki_enabled: false,
+        request_access_enabled: true
       })
 
       post api("/projects/user/#{user.id}", admin), project
@@ -887,6 +889,15 @@ describe API::API, api: true  do
         expect(json_response['message']['name']).to eq(['has already been taken'])
       end
 
+      it 'updates request_access_enabled' do
+        project_param = { request_access_enabled: false }
+
+        put api("/projects/#{project.id}", user), project_param
+
+        expect(response).to have_http_status(200)
+        expect(json_response['request_access_enabled']).to eq(false)
+      end
+
       it 'updates path & name to existing path & name in different namespace' do
         project_param = { path: project4.path, name: project4.name }
         put api("/projects/#{project3.id}", user), project_param
@@ -948,7 +959,8 @@ describe API::API, api: true  do
                           wiki_enabled: true,
                           snippets_enabled: true,
                           merge_requests_enabled: true,
-                          description: 'new description' }
+                          description: 'new description',
+                          request_access_enabled: true }
         put api("/projects/#{project.id}", user3), project_param
         expect(response).to have_http_status(403)
       end

spec/requests/ci/api/builds_spec.rb

@@ -254,7 +254,8 @@ describe Ci::API::API do
       let(:get_url) { ci_api("/builds/#{build.id}/artifacts") }
       let(:jwt_token) { JWT.encode({ 'iss' => 'gitlab-workhorse' }, Gitlab::Workhorse.secret, 'HS256') }
       let(:headers) { { "GitLab-Workhorse" => "1.0", Gitlab::Workhorse::INTERNAL_API_REQUEST_HEADER => jwt_token } }
-      let(:headers_with_token) { headers.merge(Ci::API::Helpers::BUILD_TOKEN_HEADER => build.token) }
+      let(:token) { build.token }
+      let(:headers_with_token) { headers.merge(Ci::API::Helpers::BUILD_TOKEN_HEADER => token) }
 
       before { build.run! }
 
@@ -262,6 +263,7 @@ describe Ci::API::API do
         context "should authorize posting artifact to running build" do
           it "using token as parameter" do
             post authorize_url, { token: build.token }, headers
+
             expect(response).to have_http_status(200)
             expect(response.content_type.to_s).to eq(Gitlab::Workhorse::INTERNAL_API_CONTENT_TYPE)
             expect(json_response["TempPath"]).not_to be_nil
@@ -269,6 +271,15 @@ describe Ci::API::API do
 
           it "using token as header" do
             post authorize_url, {}, headers_with_token
+
+            expect(response).to have_http_status(200)
+            expect(response.content_type.to_s).to eq(Gitlab::Workhorse::INTERNAL_API_CONTENT_TYPE)
+            expect(json_response["TempPath"]).not_to be_nil
+          end
+
+          it "using runners token" do
+            post authorize_url, { token: build.project.runners_token }, headers
+
             expect(response).to have_http_status(200)
             expect(response.content_type.to_s).to eq(Gitlab::Workhorse::INTERNAL_API_CONTENT_TYPE)
             expect(json_response["TempPath"]).not_to be_nil
@@ -276,7 +287,9 @@ describe Ci::API::API do
 
           it "reject requests that did not go through gitlab-workhorse" do
             headers.delete(Gitlab::Workhorse::INTERNAL_API_REQUEST_HEADER)
+
             post authorize_url, { token: build.token }, headers
+
             expect(response).to have_http_status(500)
           end
         end
@@ -284,13 +297,17 @@ describe Ci::API::API do
         context "should fail to post too large artifact" do
           it "using token as parameter" do
             stub_application_setting(max_artifacts_size: 0)
+
             post authorize_url, { token: build.token, filesize: 100 }, headers
+
             expect(response).to have_http_status(413)
           end
 
           it "using token as header" do
             stub_application_setting(max_artifacts_size: 0)
+
             post authorize_url, { filesize: 100 }, headers_with_token
+
             expect(response).to have_http_status(413)
           end
         end
@@ -358,6 +375,16 @@ describe Ci::API::API do
 
               it_behaves_like 'successful artifacts upload'
             end
+
+            context 'when using runners token' do
+              let(:token) { build.project.runners_token }
+
+              before do
+                upload_artifacts(file_upload, headers_with_token)
+              end
+
+              it_behaves_like 'successful artifacts upload'
+            end
           end
 
           context 'posts artifacts file and metadata file' do
@@ -497,19 +524,40 @@ describe Ci::API::API do
 
         before do
           delete delete_url, token: build.token
-          build.reload
         end
 
-        it 'removes build artifacts' do
-          expect(response).to have_http_status(200)
-          expect(build.artifacts_file.exists?).to be_falsy
-          expect(build.artifacts_metadata.exists?).to be_falsy
-          expect(build.artifacts_size).to be_nil
+        shared_examples 'having removable artifacts' do
+          it 'removes build artifacts' do
+            build.reload
+
+            expect(response).to have_http_status(200)
+            expect(build.artifacts_file.exists?).to be_falsy
+            expect(build.artifacts_metadata.exists?).to be_falsy
+            expect(build.artifacts_size).to be_nil
+          end
+        end
+
+        context 'when using build token' do
+          before do
+            delete delete_url, token: build.token
+          end
+
+          it_behaves_like 'having removable artifacts'
+        end
+
+        context 'when using runnners token' do
+          before do
+            delete delete_url, token: build.project.runners_token
+          end
+
+          it_behaves_like 'having removable artifacts'
         end
       end
 
       describe 'GET /builds/:id/artifacts' do
-        before { get get_url, token: build.token }
+        before do
+          get get_url, token: token
+        end
 
         context 'build has artifacts' do
           let(:build) { create(:ci_build, :artifacts) }
@@ -518,13 +566,29 @@ describe Ci::API::API do
               'Content-Disposition' => 'attachment; filename=ci_build_artifacts.zip' }
           end
 
-          it 'downloads artifact' do
-            expect(response).to have_http_status(200)
-            expect(response.headers).to include download_headers
+          shared_examples 'having downloadable artifacts' do
+            it 'download artifacts' do
+              expect(response).to have_http_status(200)
+              expect(response.headers).to include download_headers
+            end
+          end
+
+          context 'when using build token' do
+            let(:token) { build.token }
+
+            it_behaves_like 'having downloadable artifacts'
+          end
+
+          context 'when using runnners token' do
+            let(:token) { build.project.runners_token }
+
+            it_behaves_like 'having downloadable artifacts'
           end
         end
 
         context 'build does not has artifacts' do
+          let(:token) { build.token }
+
           it 'responds with not found' do
             expect(response).to have_http_status(404)
           end

spec/requests/git_http_spec.rb

@@ -300,25 +300,79 @@ describe 'Git HTTP requests', lib: true do
       end
 
       context "when a gitlab ci token is provided" do
-        let(:token) { 123 }
-        let(:project) { FactoryGirl.create :empty_project }
+        let(:build) { create(:ci_build, :running) }
+        let(:project) { build.project }
+        let(:other_project) { create(:empty_project) }
 
         before do
-          project.update_attributes(runners_token: token)
           project.project_feature.update_attributes(builds_access_level: ProjectFeature::ENABLED)
         end
 
-        it "downloads get status 200" do
-          clone_get "#{project.path_with_namespace}.git", user: 'gitlab-ci-token', password: token
+        context 'when build created by system is authenticated' do
+          it "downloads get status 200" do
+            clone_get "#{project.path_with_namespace}.git", user: 'gitlab-ci-token', password: build.token
 
-          expect(response).to have_http_status(200)
-          expect(response.content_type.to_s).to eq(Gitlab::Workhorse::INTERNAL_API_CONTENT_TYPE)
+            expect(response).to have_http_status(200)
+            expect(response.content_type.to_s).to eq(Gitlab::Workhorse::INTERNAL_API_CONTENT_TYPE)
+          end
+
+          it "uploads get status 401 (no project existence information leak)" do
+            push_get "#{project.path_with_namespace}.git", user: 'gitlab-ci-token', password: build.token
+
+            expect(response).to have_http_status(401)
+          end
+
+          it "downloads from other project get status 404" do
+            clone_get "#{other_project.path_with_namespace}.git", user: 'gitlab-ci-token', password: build.token
+
+            expect(response).to have_http_status(404)
+          end
         end
 
-        it "uploads get status 401 (no project existence information leak)" do
-          push_get "#{project.path_with_namespace}.git", user: 'gitlab-ci-token', password: token
+        context 'and build created by' do
+          before do
+            build.update(user: user)
+            project.team << [user, :reporter]
+          end
 
-          expect(response).to have_http_status(401)
+          shared_examples 'can download code only from own projects' do
+            it 'downloads get status 200' do
+              clone_get "#{project.path_with_namespace}.git", user: 'gitlab-ci-token', password: build.token
+
+              expect(response).to have_http_status(200)
+              expect(response.content_type.to_s).to eq(Gitlab::Workhorse::INTERNAL_API_CONTENT_TYPE)
+            end
+
+            it 'uploads get status 403' do
+              push_get "#{project.path_with_namespace}.git", user: 'gitlab-ci-token', password: build.token
+
+              expect(response).to have_http_status(401)
+            end
+          end
+
+          context 'administrator' do
+            let(:user) { create(:admin) }
+
+            it_behaves_like 'can download code only from own projects'
+
+            it 'downloads from other project get status 403' do
+              clone_get "#{other_project.path_with_namespace}.git", user: 'gitlab-ci-token', password: build.token
+
+              expect(response).to have_http_status(403)
+            end
+          end
+
+          context 'regular user' do
+            let(:user) { create(:user) }
+
+            it_behaves_like 'can download code only from own projects'
+
+            it 'downloads from other project get status 404' do
+              clone_get "#{other_project.path_with_namespace}.git", user: 'gitlab-ci-token', password: build.token
+
+              expect(response).to have_http_status(404)
+            end
+          end
         end
       end
     end

spec/requests/jwt_controller_spec.rb

@@ -22,11 +22,13 @@ describe JwtController do
 
   context 'when using authorized request' do
     context 'using CI token' do
-      let(:project) { create(:empty_project, runners_token: 'token') }
-      let(:headers) { { authorization: credentials('gitlab-ci-token', project.runners_token) } }
+      let(:build) { create(:ci_build, :running) }
+      let(:project) { build.project }
+      let(:headers) { { authorization: credentials('gitlab-ci-token', build.token) } }
 
       context 'project with enabled CI' do
         subject! { get '/jwt/auth', parameters, headers }
+
         it { expect(service_class).to have_received(:new).with(project, nil, parameters) }
       end
 
@@ -43,13 +45,31 @@ describe JwtController do
 
     context 'using User login' do
       let(:user) { create(:user) }
-      let(:headers) { { authorization: credentials('user', 'password') } }
-
-      before { expect(Gitlab::Auth).to receive(:find_with_user_password).with('user', 'password').and_return(user) }
+      let(:headers) { { authorization: credentials(user.username, user.password) } }
 
       subject! { get '/jwt/auth', parameters, headers }
 
       it { expect(service_class).to have_received(:new).with(nil, user, parameters) }
+
+      context 'when user has 2FA enabled' do
+        let(:user) { create(:user, :two_factor) }
+
+        context 'without personal token' do
+          it 'rejects the authorization attempt' do
+            expect(response).to have_http_status(401)
+            expect(response.body).to include('You have 2FA enabled, please use a personal access token for Git over HTTP')
+          end
+        end
+
+        context 'with personal token' do
+          let(:access_token) { create(:personal_access_token, user: user) }
+          let(:headers) { { authorization: credentials(user.username, access_token.token) } }
+
+          it 'rejects the authorization attempt' do
+            expect(response).to have_http_status(200)
+          end
+        end
+      end
     end
 
     context 'using invalid login' do

spec/requests/lfs_http_spec.rb

@@ -14,6 +14,7 @@ describe 'Git LFS API and storage' do
   end
   let(:authorization) { }
   let(:sendfile) { }
+  let(:pipeline) { create(:ci_empty_pipeline, project: project) }
 
   let(:sample_oid) { lfs_object.oid }
   let(:sample_size) { lfs_object.size }
@@ -244,14 +245,63 @@ describe 'Git LFS API and storage' do
           end
         end
 
-        context 'when CI is authorized' do
+        context 'when build is authorized as' do
           let(:authorization) { authorize_ci_project }
 
-          let(:update_permissions) do
-            project.lfs_objects << lfs_object
+          shared_examples 'can download LFS only from own projects' do
+            context 'for own project' do
+              let(:pipeline) { create(:ci_empty_pipeline, project: project) }
+
+              let(:update_permissions) do
+                project.team << [user, :reporter]
+                project.lfs_objects << lfs_object
+              end
+
+              it_behaves_like 'responds with a file'
+            end
+
+            context 'for other project' do
+              let(:other_project) { create(:empty_project) }
+              let(:pipeline) { create(:ci_empty_pipeline, project: other_project) }
+
+              let(:update_permissions) do
+                project.lfs_objects << lfs_object
+              end
+
+              it 'rejects downloading code' do
+                expect(response).to have_http_status(other_project_status)
+              end
+            end
+          end
+
+          context 'administrator' do
+            let(:user) { create(:admin) }
+            let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+            it_behaves_like 'can download LFS only from own projects' do
+              # We render 403, because administrator does have normally access
+              let(:other_project_status) { 403 }
+            end
+          end
+
+          context 'regular user' do
+            let(:user) { create(:user) }
+            let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+            it_behaves_like 'can download LFS only from own projects' do
+              # We render 404, to prevent data leakage about existence of the project
+              let(:other_project_status) { 404 }
+            end
           end
 
-          it_behaves_like 'responds with a file'
+          context 'does not have user' do
+            let(:build) { create(:ci_build, :running, pipeline: pipeline) }
+
+            it_behaves_like 'can download LFS only from own projects' do
+              # We render 404, to prevent data leakage about existence of the project
+              let(:other_project_status) { 404 }
+            end
+          end
         end
       end
 
@@ -431,10 +481,62 @@ describe 'Git LFS API and storage' do
         end
       end
 
-      context 'when CI is authorized' do
+      context 'when build is authorized as' do
         let(:authorization) { authorize_ci_project }
 
-        it_behaves_like 'an authorized requests'
+        let(:update_lfs_permissions) do
+          project.lfs_objects << lfs_object
+        end
+
+        shared_examples 'can download LFS only from own projects' do
+          context 'for own project' do
+            let(:pipeline) { create(:ci_empty_pipeline, project: project) }
+
+            let(:update_user_permissions) do
+              project.team << [user, :reporter]
+            end
+
+            it_behaves_like 'an authorized requests'
+          end
+
+          context 'for other project' do
+            let(:other_project) { create(:empty_project) }
+            let(:pipeline) { create(:ci_empty_pipeline, project: other_project) }
+
+            it 'rejects downloading code' do
+              expect(response).to have_http_status(other_project_status)
+            end
+          end
+        end
+
+        context 'administrator' do
+          let(:user) { create(:admin) }
+          let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+          it_behaves_like 'can download LFS only from own projects' do
+            # We render 403, because administrator does have normally access
+            let(:other_project_status) { 403 }
+          end
+        end
+
+        context 'regular user' do
+          let(:user) { create(:user) }
+          let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+          it_behaves_like 'can download LFS only from own projects' do
+            # We render 404, to prevent data leakage about existence of the project
+            let(:other_project_status) { 404 }
+          end
+        end
+
+        context 'does not have user' do
+          let(:build) { create(:ci_build, :running, pipeline: pipeline) }
+
+          it_behaves_like 'can download LFS only from own projects' do
+            # We render 404, to prevent data leakage about existence of the project
+            let(:other_project_status) { 404 }
+          end
+        end
       end
 
       context 'when user is not authenticated' do
@@ -583,11 +685,37 @@ describe 'Git LFS API and storage' do
           end
         end
 
-        context 'when CI is authorized' do
+        context 'when build is authorized' do
           let(:authorization) { authorize_ci_project }
 
-          it 'responds with 401' do
-            expect(response).to have_http_status(401)
+          context 'build has an user' do
+            let(:user) { create(:user) }
+
+            context 'tries to push to own project' do
+              let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+              it 'responds with 401' do
+                expect(response).to have_http_status(401)
+              end
+            end
+
+            context 'tries to push to other project' do
+              let(:other_project) { create(:empty_project) }
+              let(:pipeline) { create(:ci_empty_pipeline, project: other_project) }
+              let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+              it 'responds with 401' do
+                expect(response).to have_http_status(401)
+              end
+            end
+          end
+
+          context 'does not have user' do
+            let(:build) { create(:ci_build, :running, pipeline: pipeline) }
+
+            it 'responds with 401' do
+              expect(response).to have_http_status(401)
+            end
           end
         end
       end
@@ -609,14 +737,6 @@ describe 'Git LFS API and storage' do
           end
         end
       end
-
-      context 'when CI is authorized' do
-        let(:authorization) { authorize_ci_project }
-
-        it 'responds with status 403' do
-          expect(response).to have_http_status(401)
-        end
-      end
     end
 
     describe 'unsupported' do
@@ -779,10 +899,51 @@ describe 'Git LFS API and storage' do
         end
       end
 
-      context 'when CI is authenticated' do
+      context 'when build is authorized' do
         let(:authorization) { authorize_ci_project }
 
-        it_behaves_like 'unauthorized'
+        context 'build has an user' do
+          let(:user) { create(:user) }
+
+          context 'tries to push to own project' do
+            let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+            before do
+              project.team << [user, :developer]
+              put_authorize
+            end
+
+            it 'responds with 401' do
+              expect(response).to have_http_status(401)
+            end
+          end
+
+          context 'tries to push to other project' do
+            let(:other_project) { create(:empty_project) }
+            let(:pipeline) { create(:ci_empty_pipeline, project: other_project) }
+            let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+            before do
+              put_authorize
+            end
+
+            it 'responds with 401' do
+              expect(response).to have_http_status(401)
+            end
+          end
+        end
+
+        context 'does not have user' do
+          let(:build) { create(:ci_build, :running, pipeline: pipeline) }
+
+          before do
+            put_authorize
+          end
+
+          it 'responds with 401' do
+            expect(response).to have_http_status(401)
+          end
+        end
       end
 
       context 'for unauthenticated' do
@@ -839,10 +1000,42 @@ describe 'Git LFS API and storage' do
         end
       end
 
-      context 'when CI is authenticated' do
+      context 'when build is authorized' do
         let(:authorization) { authorize_ci_project }
 
-        it_behaves_like 'unauthorized'
+        before do
+          put_authorize
+        end
+
+        context 'build has an user' do
+          let(:user) { create(:user) }
+
+          context 'tries to push to own project' do
+            let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+            it 'responds with 401' do
+              expect(response).to have_http_status(401)
+            end
+          end
+
+          context 'tries to push to other project' do
+            let(:other_project) { create(:empty_project) }
+            let(:pipeline) { create(:ci_empty_pipeline, project: other_project) }
+            let(:build) { create(:ci_build, :running, pipeline: pipeline, user: user) }
+
+            it 'responds with 401' do
+              expect(response).to have_http_status(401)
+            end
+          end
+        end
+
+        context 'does not have user' do
+          let(:build) { create(:ci_build, :running, pipeline: pipeline) }
+
+          it 'responds with 401' do
+            expect(response).to have_http_status(401)
+          end
+        end
       end
 
       context 'for unauthenticated' do
@@ -897,7 +1090,7 @@ describe 'Git LFS API and storage' do
   end
 
   def authorize_ci_project
-    ActionController::HttpAuthentication::Basic.encode_credentials('gitlab-ci-token', project.runners_token)
+    ActionController::HttpAuthentication::Basic.encode_credentials('gitlab-ci-token', build.token)
   end
 
   def authorize_user

spec/services/auth/container_registry_authentication_service_spec.rb

@@ -6,8 +6,14 @@ describe Auth::ContainerRegistryAuthenticationService, services: true do
   let(:current_params) { {} }
   let(:rsa_key) { OpenSSL::PKey::RSA.generate(512) }
   let(:payload) { JWT.decode(subject[:token], rsa_key).first }
+  let(:authentication_abilities) do
+    [
+      :read_container_image,
+      :create_container_image
+    ]
+  end
 
-  subject { described_class.new(current_project, current_user, current_params).execute }
+  subject { described_class.new(current_project, current_user, current_params).execute(authentication_abilities: authentication_abilities) }
 
   before do
     allow(Gitlab.config.registry).to receive_messages(enabled: true, issuer: 'rspec', key: nil)
@@ -189,13 +195,22 @@ describe Auth::ContainerRegistryAuthenticationService, services: true do
     end
   end
 
-  context 'project authorization' do
+  context 'build authorized as user' do
     let(:current_project) { create(:empty_project) }
+    let(:current_user) { create(:user) }
+    let(:authentication_abilities) do
+      [
+        :build_read_container_image,
+        :build_create_container_image
+      ]
+    end
 
-    context 'allow to use scope-less authentication' do
-      it_behaves_like 'a valid token'
+    before do
+      current_project.team << [current_user, :developer]
     end
 
+    it_behaves_like 'a valid token'
+
     context 'allow to pull and push images' do
       let(:current_params) do
         { scope: "repository:#{current_project.path_with_namespace}:pull,push" }
@@ -214,12 +229,44 @@ describe Auth::ContainerRegistryAuthenticationService, services: true do
 
         context 'allow for public' do
           let(:project) { create(:empty_project, :public) }
+
           it_behaves_like 'a pullable'
         end
 
-        context 'disallow for private' do
+        shared_examples 'pullable for being team member' do
+          context 'when you are not member' do
+            it_behaves_like 'an inaccessible'
+          end
+
+          context 'when you are member' do
+            before do
+              project.team << [current_user, :developer]
+            end
+
+            it_behaves_like 'a pullable'
+          end
+        end
+
+        context 'for private' do
           let(:project) { create(:empty_project, :private) }
-          it_behaves_like 'an inaccessible'
+
+          it_behaves_like 'pullable for being team member'
+
+          context 'when you are admin' do
+            let(:current_user) { create(:admin) }
+
+            context 'when you are not member' do
+              it_behaves_like 'an inaccessible'
+            end
+
+            context 'when you are member' do
+              before do
+                project.team << [current_user, :developer]
+              end
+
+              it_behaves_like 'a pullable'
+            end
+          end
         end
       end
 
@@ -230,6 +277,11 @@ describe Auth::ContainerRegistryAuthenticationService, services: true do
 
         context 'disallow for all' do
           let(:project) { create(:empty_project, :public) }
+
+          before do
+            project.team << [current_user, :developer]
+          end
+
           it_behaves_like 'an inaccessible'
         end
       end

spec/services/create_deployment_service_spec.rb

@@ -41,7 +41,7 @@ describe CreateDeploymentService, services: true do
 
     context 'for environment with invalid name' do
       let(:params) do
-        { environment: 'name with spaces',
+        { environment: 'name,with,commas',
           ref: 'master',
           tag: false,
           sha: '97de212e80737a608d939f648d959671fb0a0142',
@@ -56,8 +56,36 @@ describe CreateDeploymentService, services: true do
         expect(subject).not_to be_persisted
       end
     end
+
+    context 'when variables are used' do
+      let(:params) do
+        { environment: 'review-apps/$CI_BUILD_REF_NAME',
+          ref: 'master',
+          tag: false,
+          sha: '97de212e80737a608d939f648d959671fb0a0142',
+          options: {
+            name: 'review-apps/$CI_BUILD_REF_NAME',
+            url: 'http://$CI_BUILD_REF_NAME.review-apps.gitlab.com'
+          },
+          variables: [
+            { key: 'CI_BUILD_REF_NAME', value: 'feature-review-apps' }
+          ]
+        }
+      end
+
+      it 'does create a new environment' do
+        expect { subject }.to change { Environment.count }.by(1)
+
+        expect(subject.environment.name).to eq('review-apps/feature-review-apps')
+        expect(subject.environment.external_url).to eq('http://feature-review-apps.review-apps.gitlab.com')
+      end
+
+      it 'does create a new deployment' do
+        expect(subject).to be_persisted
+      end
+    end
   end
-  
+
   describe 'processing of builds' do
     let(:environment) { nil }
     
@@ -95,6 +123,12 @@ describe CreateDeploymentService, services: true do
 
         expect(Deployment.last.deployable).to eq(deployable)
       end
+
+      it 'create environment has URL set' do
+        subject
+
+        expect(Deployment.last.environment.external_url).not_to be_nil
+      end
     end
 
     context 'without environment specified' do
@@ -107,7 +141,10 @@ describe CreateDeploymentService, services: true do
     
     context 'when environment is specified' do
       let(:pipeline) { create(:ci_pipeline, project: project) }
-      let(:build) { create(:ci_build, pipeline: pipeline, environment: 'production') }
+      let(:build) { create(:ci_build, pipeline: pipeline, environment: 'production', options: options) }
+      let(:options) do
+        { environment: { name: 'production', url: 'http://gitlab.com' } }
+      end
 
       context 'when build succeeds' do
         it_behaves_like 'does create environment and deployment' do

spec/services/git_push_service_spec.rb

@@ -253,6 +253,21 @@ describe GitPushService, services: true do
         expect(project.protected_branches.last.merge_access_levels.map(&:access_level)).to eq([Gitlab::Access::MASTER])
       end
 
+      it "when pushing a branch for the first time with an existing branch permission configured" do
+        stub_application_setting(default_branch_protection: Gitlab::Access::PROTECTION_DEV_CAN_PUSH)
+
+        create(:protected_branch, :no_one_can_push, :developers_can_merge, project: project, name: 'master')
+        expect(project).to receive(:execute_hooks)
+        expect(project.default_branch).to eq("master")
+        expect_any_instance_of(ProtectedBranches::CreateService).not_to receive(:execute)
+
+        execute_service(project, user, @blankrev, 'newrev', 'refs/heads/master' )
+
+        expect(project.protected_branches).not_to be_empty
+        expect(project.protected_branches.last.push_access_levels.map(&:access_level)).to eq([Gitlab::Access::NO_ACCESS])
+        expect(project.protected_branches.last.merge_access_levels.map(&:access_level)).to eq([Gitlab::Access::DEVELOPER])
+      end
+
       it "when pushing a branch for the first time with default branch protection set to 'developers can merge'" do
         stub_application_setting(default_branch_protection: Gitlab::Access::PROTECTION_DEV_CAN_MERGE)