Merge branch 'sh-security-10-0-stable-ee' into 'master'

Backport all fixes from GitLab 10.1 EE into master

See merge request gitlab-org/gitlab-ee!3164

CHANGELOG-EE.md

 Please view this file on the master branch, on stable branches it's out of date.
 
+## 10.0.4 (2017-10-16)
+
+- [SECURITY] Prevent Related Issues from leaking confidential issues. !541
+- [SECURITY] Escape user name in filtered search bar.
+
 ## 10.0.3 (2017-10-05)
 
 - [FIXED] Rewrite Geo database rake tasks so they operate on the correct database. !3052
@@ -61,6 +66,11 @@ Please view this file on the master branch, on stable branches it's out of date.
 - Add group issue boards.
 - Ports style changes fixed in a conflict in ce to ee upstream to master for new projects page.
 
+## 9.5.9 (2017-10-16)
+
+- [SECURITY] Prevent Related Issues from leaking confidential issues.
+- Escape user name in filtered search bar.
+
 ## 9.5.8 (2017-10-04)
 
 - [FIXED] Fix EE delta size check handling with annotated tags.
@@ -137,6 +147,12 @@ Please view this file on the master branch, on stable branches it's out of date.
 - Fix rebase button when merge request is created from a fork.
 - Skip oAuth authorization for trusted applications.
 
+## 9.4.7 (2017-10-16)
+
+- [SECURITY] Prevent Related Issues from leaking confidential issues.
+- Fix when pushing without a branch name. !2879
+- Escape user name in filtered search bar.
+
 ## 9.4.6 (2017-09-06)
 
 - [FIXED] Validate branch name push rule when pushing branch without commits. !2685

CHANGELOG.md

@@ -2,6 +2,12 @@
 documentation](doc/development/changelog.md) for instructions on adding your own
 entry.
 
+## 10.0.4 (2017-10-16)
+
+- [SECURITY] Move project repositories between namespaces when renaming users.
+- [SECURITY] Prevent an open redirect on project pages.
+- [SECURITY] Prevent a persistent XSS in user-provided markup.
+
 ## 10.0.3 (2017-10-05)
 
 - [FIXED] find_user Users helper method no longer overrides find_user API helper method. !14418
@@ -212,6 +218,14 @@ entry.
 - Added type to CHANGELOG entries. (Jacopo Beschi @jacopo-beschi)
 - [BUGIFX] Improves subgroup creation permissions. !13418
 
+## 9.5.9 (2017-10-16)
+
+- [SECURITY] Move project repositories between namespaces when renaming users.
+- [SECURITY] Prevent an open redirect on project pages.
+- [SECURITY] Prevent a persistent XSS in user-provided markup.
+- [FIXED] Allow using newlines in pipeline email service recipients. !14250
+- Escape user name in filtered search bar.
+
 ## 9.5.8 (2017-10-04)
 
 - [FIXED] Fixed fork button being disabled for users who can fork to a group.
@@ -457,6 +471,15 @@ entry.
 - Use a specialized class for querying events to improve performance.
 - Update build badges to be pipeline badges and display passing instead of success.
 
+## 9.4.7 (2017-10-16)
+
+- [SECURITY] Upgrade mail and nokogiri gems due to security issues. !13662 (Markus Koller)
+- [SECURITY] Move project repositories between namespaces when renaming users.
+- [SECURITY] Prevent an open redirect on project pages.
+- [SECURITY] Prevent a persistent XSS in user-provided markup.
+- [FIXED] Allow using newlines in pipeline email service recipients. !14250
+- Escape user name in filtered search bar.
+
 ## 9.4.6 (2017-09-06)
 
 - [SECURITY] Upgrade mail and nokogiri gems due to security issues. !13662 (Markus Koller)

app/assets/javascripts/filtered_search/filtered_search_visual_tokens.js

@@ -123,8 +123,8 @@ class FilteredSearchVisualTokens {
         /* eslint-disable no-param-reassign */
         tokenValueContainer.dataset.originalValue = tokenValue;
         tokenValueElement.innerHTML = `
-          <img class="avatar s20" src="${user.avatar_url}" alt="${user.name}'s avatar">
-          ${user.name}
+          <img class="avatar s20" src="${user.avatar_url}" alt="">
+          ${_.escape(user.name)}
         `;
         /* eslint-enable no-param-reassign */
       })

app/controllers/projects/application_controller.rb

@@ -2,7 +2,6 @@ class Projects::ApplicationController < ApplicationController
   include RoutableActions
 
   skip_before_action :authenticate_user!
-  before_action :redirect_git_extension
   before_action :project
   before_action :repository
   layout 'project'
@@ -11,15 +10,6 @@ class Projects::ApplicationController < ApplicationController
 
   private
 
-  def redirect_git_extension
-    # Redirect from
-    #   localhost/group/project.git
-    # to
-    #   localhost/group/project
-    #
-    redirect_to url_for(params.merge(format: nil)) if params[:format] == 'git'
-  end
-
   def project
     return @project if @project
     return nil unless params[:project_id] || params[:id]

app/controllers/projects_controller.rb

@@ -5,6 +5,7 @@ class ProjectsController < Projects::ApplicationController
   prepend EE::ProjectsController
 
   before_action :authenticate_user!, except: [:index, :show, :activity, :refs]
+  before_action :redirect_git_extension, only: [:show]
   before_action :project, except: [:index, :new, :create]
   before_action :repository, except: [:index, :new, :create]
   before_action :assign_ref_vars, only: [:show], if: :repo_exists?
@@ -399,4 +400,13 @@ class ProjectsController < Projects::ApplicationController
   def project_export_enabled
     render_404 unless current_application_settings.project_export_enabled?
   end
+
+  def redirect_git_extension
+    # Redirect from
+    #   localhost/group/project.git
+    # to
+    #   localhost/group/project
+    #
+    redirect_to request.original_url.sub(/\.git\/?\Z/, '') if params[:format] == 'git'
+  end
 end

app/models/concerns/storage/legacy_namespace.rb

@@ -7,6 +7,8 @@ module Storage
         raise Gitlab::UpdatePathError.new('Namespace cannot be moved, because at least one project has tags in container registry')
       end
 
+      expires_full_path_cache
+
       # Move the namespace directory in all storage paths used by member projects
       repository_storage_paths.each do |repository_storage_path|
         # Ensure old directory exists before moving it

app/models/note.rb

@@ -175,7 +175,7 @@ class Note < ActiveRecord::Base
   end
 
   def cross_reference?
-    system? && SystemNoteService.cross_reference?(note)
+    system? && matches_cross_reference_regex?
   end
 
   def diff_note?

app/services/system_note_service.rb

@@ -162,7 +162,6 @@ module SystemNoteService
   #   "changed time estimate to 3d 5h"
   #
   # Returns the created Note object
-
   def change_time_estimate(noteable, project, author)
     parsed_time = Gitlab::TimeTrackingFormatter.output(noteable.time_estimate)
     body = if noteable.time_estimate == 0
@@ -188,7 +187,6 @@ module SystemNoteService
   #   "added 2h 30m of time spent"
   #
   # Returns the created Note object
-
   def change_time_spent(noteable, project, author)
     time_spent = noteable.time_spent
 
@@ -453,10 +451,6 @@ module SystemNoteService
     end
   end
 
-  def cross_reference?(note_text)
-    note_text =~ /\A#{cross_reference_note_prefix}/i
-  end
-
   # Check if a cross-reference is disallowed
   #
   # This method prevents adding a "mentioned in !1" note on every single commit
@@ -486,7 +480,6 @@ module SystemNoteService
   # mentioner - Mentionable object
   #
   # Returns Boolean
-
   def cross_reference_exists?(noteable, mentioner)
     # Initial scope should be system notes of this noteable type
     notes = Note.system.where(noteable_type: noteable.class)

lib/banzai/filter/sanitization_filter.rb

@@ -75,9 +75,19 @@ module Banzai
             begin
               node['href'] = node['href'].strip
               uri = Addressable::URI.parse(node['href'])
-              uri.scheme = uri.scheme.downcase if uri.scheme
 
-              node.remove_attribute('href') if UNSAFE_PROTOCOLS.include?(uri.scheme)
+              return unless uri.scheme
+
+              # Remove all invalid scheme characters before checking against the
+              # list of unsafe protocols.
+              #
+              # See https://tools.ietf.org/html/rfc3986#section-3.1
+              scheme = uri.scheme
+                .strip
+                .downcase
+                .gsub(/[^A-Za-z0-9\+\.\-]+/, '')
+
+              node.remove_attribute('href') if UNSAFE_PROTOCOLS.include?(scheme)
             rescue Addressable::URI::InvalidURIError
               node.remove_attribute('href')
             end

spec/controllers/profiles_controller_spec.rb

 require('spec_helper')
 
-describe ProfilesController do
-  describe "PUT update" do
-    it "allows an email update from a user without an external email address" do
-      user = create(:user)
+describe ProfilesController, :request_store do
+  let(:user) { create(:user) }
+
+  describe 'PUT update' do
+    it 'allows an email update from a user without an external email address' do
       sign_in(user)
 
       put :update,
@@ -29,7 +30,7 @@ describe ProfilesController do
       expect(user.unconfirmed_email).to eq nil
     end
 
-    it "ignores an email update from a user with an external email address" do
+    it 'ignores an email update from a user with an external email address' do
       stub_omniauth_setting(sync_profile_from_provider: ['ldap'])
       stub_omniauth_setting(sync_profile_attributes: true)
 
@@ -46,7 +47,7 @@ describe ProfilesController do
       expect(ldap_user.unconfirmed_email).not_to eq('john@gmail.com')
     end
 
-    it "ignores an email and name update but allows a location update from a user with external email and name, but not external location" do
+    it 'ignores an email and name update but allows a location update from a user with external email and name, but not external location' do
       stub_omniauth_setting(sync_profile_from_provider: ['ldap'])
       stub_omniauth_setting(sync_profile_attributes: true)
 
@@ -65,4 +66,35 @@ describe ProfilesController do
       expect(ldap_user.location).to eq('City, Country')
     end
   end
+
+  describe 'PUT update_username' do
+    let(:namespace) { user.namespace }
+    let(:project) { create(:project_empty_repo, namespace: namespace) }
+    let(:gitlab_shell) { Gitlab::Shell.new }
+    let(:new_username) { 'renamedtosomethingelse' }
+
+    it 'allows username change' do
+      sign_in(user)
+
+      put :update_username,
+        user: { username: new_username }
+
+      user.reload
+
+      expect(response.status).to eq(302)
+      expect(user.username).to eq(new_username)
+    end
+
+    it 'moves dependent projects to new namespace' do
+      sign_in(user)
+
+      put :update_username,
+        user: { username: new_username }
+
+      user.reload
+
+      expect(response.status).to eq(302)
+      expect(gitlab_shell.exists?(project.repository_storage_path, "#{new_username}/#{project.path}.git")).to be_truthy
+    end
+  end
 end

spec/controllers/projects/issues_controller_spec.rb

@@ -850,47 +850,107 @@ describe Projects::IssuesController do
 
   describe 'GET #discussions' do
     let!(:discussion) { create(:discussion_note_on_issue, noteable: issue, project: issue.project) }
+    context 'when authenticated' do
+      before do
+        project.add_developer(user)
+        sign_in(user)
+      end
 
-    before do
-      project.add_developer(user)
-      sign_in(user)
-    end
+      it 'returns discussion json' do
+        get :discussions, namespace_id: project.namespace, project_id: project, id: issue.iid
 
-    it 'returns discussion json' do
-      get :discussions, namespace_id: project.namespace, project_id: project, id: issue.iid
+        expect(json_response.first.keys).to match_array(%w[id reply_id expanded notes individual_note])
+      end
 
-      expect(JSON.parse(response.body).first.keys).to match_array(%w[id reply_id expanded notes individual_note])
-    end
+      context 'with cross-reference system note', :request_store do
+        let(:new_issue) { create(:issue) }
+        let(:cross_reference) { "mentioned in #{new_issue.to_reference(issue.project)}" }
 
-    context 'with cross-reference system note', :request_store do
-      let(:new_issue) { create(:issue) }
-      let(:cross_reference) { "mentioned in #{new_issue.to_reference(issue.project)}" }
+        before do
+          create(:discussion_note_on_issue, :system, noteable: issue, project: issue.project, note: cross_reference)
+        end
 
-      before do
-        create(:discussion_note_on_issue, :system, noteable: issue, project: issue.project, note: cross_reference)
+        it 'filters notes that the user should not see' do
+          get :discussions, namespace_id: project.namespace, project_id: project, id: issue.iid
+
+          expect(JSON.parse(response.body).count).to eq(1)
+        end
+
+        it 'does not result in N+1 queries' do
+          # Instantiate the controller variables to ensure QueryRecorder has an accurate base count
+          get :discussions, namespace_id: project.namespace, project_id: project, id: issue.iid
+
+          RequestStore.clear!
+
+          control_count = ActiveRecord::QueryRecorder.new do
+            get :discussions, namespace_id: project.namespace, project_id: project, id: issue.iid
+          end.count
+
+          RequestStore.clear!
+
+          create_list(:discussion_note_on_issue, 2, :system, noteable: issue, project: issue.project, note: cross_reference)
+
+          expect { get :discussions, namespace_id: project.namespace, project_id: project, id: issue.iid }.not_to exceed_query_limit(control_count)
+        end
       end
+    end
 
-      it 'filters notes that the user should not see' do
-        get :discussions, namespace_id: project.namespace, project_id: project, id: issue.iid
+    context 'with a related system note' do
+      let(:confidential_issue) { create(:issue, :confidential, project: project) }
+      let!(:system_note) { SystemNoteService.relate_issue(issue, confidential_issue, user) }
+
+      shared_examples 'user can see confidential issue' do |access_level|
+        context "when a user is a #{access_level}" do
+          before do
+            project.add_user(user, access_level)
+          end
+
+          it 'displays related notes' do
+            get :discussions, namespace_id: project.namespace, project_id: project, id: issue.iid
 
-        expect(JSON.parse(response.body).count).to eq(1)
+            discussions = json_response
+            notes = discussions.flat_map {|d| d['notes']}
+
+            expect(discussions.count).to equal(2)
+            expect(notes).to include(a_hash_including('id' => system_note.id))
+          end
+        end
       end
 
-      it 'does not result in N+1 queries' do
-        # Instantiate the controller variables to ensure QueryRecorder has an accurate base count
-        get :discussions, namespace_id: project.namespace, project_id: project, id: issue.iid
+      shared_examples 'user cannot see confidential issue' do |access_level|
+        context "when a user is a #{access_level}" do
+          before do
+            project.add_user(user, access_level)
+          end
 
-        RequestStore.clear!
+          it 'redacts note related to a confidential issue' do
+            get :discussions, namespace_id: project.namespace, project_id: project, id: issue.iid
 
-        control_count = ActiveRecord::QueryRecorder.new do
-          get :discussions, namespace_id: project.namespace, project_id: project, id: issue.iid
-        end.count
+            discussions = json_response
+            notes = discussions.flat_map {|d| d['notes']}
+
+            expect(discussions.count).to equal(1)
+            expect(notes).not_to include(a_hash_including('id' => system_note.id))
+          end
+        end
+      end
+
+      context 'when authenticated' do
+        before do
+          sign_in(user)
+        end
 
-        RequestStore.clear!
+        %i(reporter developer master).each do |access|
+          it_behaves_like 'user can see confidential issue', access
+        end
+
+        it_behaves_like 'user cannot see confidential issue', :guest
+      end
 
-        create_list(:discussion_note_on_issue, 2, :system, noteable: issue, project: issue.project, note: cross_reference)
+      context 'when unauthenticated' do
+        let(:project) { create(:project, :public) }
 
-        expect { get :discussions, namespace_id: project.namespace, project_id: project, id: issue.iid }.not_to exceed_query_limit(control_count)
+        it_behaves_like 'user cannot see confidential issue', Gitlab::Access::NO_ACCESS
       end
     end
   end

spec/javascripts/filtered_search/filtered_search_visual_tokens_spec.js

@@ -791,6 +791,29 @@ describe('Filtered Search Visual Tokens', () => {
         expect(tokenValueElement.innerText.trim()).toBe(dummyUser.name);
         const avatar = tokenValueElement.querySelector('img.avatar');
         expect(avatar.src).toBe(dummyUser.avatar_url);
+        expect(avatar.alt).toBe('');
+      })
+      .then(done)
+      .catch(done.fail);
+    });
+
+    it('escapes user name when creating token', (done) => {
+      const dummyUser = {
+        name: '<script>',
+        avatar_url: `${gl.TEST_HOST}/mypics/avatar.png`,
+      };
+      const { tokenValueContainer, tokenValueElement } = findElements(authorToken);
+      const tokenValue = tokenValueElement.innerText;
+      usersCacheSpy = (username) => {
+        expect(`@${username}`).toBe(tokenValue);
+        return Promise.resolve(dummyUser);
+      };
+
+      subject.updateUserTokenAppearance(tokenValueContainer, tokenValueElement, tokenValue)
+      .then(() => {
+        expect(tokenValueElement.innerText.trim()).toBe(dummyUser.name);
+        tokenValueElement.querySelector('.avatar').remove();
+        expect(tokenValueElement.innerHTML.trim()).toBe(_.escape(dummyUser.name));
       })
       .then(done)
       .catch(done.fail);

spec/lib/banzai/filter/sanitization_filter_spec.rb

@@ -217,6 +217,11 @@ describe Banzai::Filter::SanitizationFilter do
         output: '<img>'
       },
 
+      'protocol-based JS injection: Unicode' => {
+        input: %Q(<a href="\u0001java\u0003script:alert('XSS')">foo</a>),
+        output: '<a>foo</a>'
+      },
+
       'protocol-based JS injection: spaces and entities' => {
         input:  '<a href=" &#14;  javascript:alert(\'XSS\');">foo</a>',
         output: '<a href="">foo</a>'

spec/models/namespace_spec.rb

@@ -4,6 +4,7 @@ describe Namespace do
   include ProjectForksHelper
 
   let!(:namespace) { create(:namespace) }
+  let(:gitlab_shell) { Gitlab::Shell.new }
 
   describe 'associations' do
     it { is_expected.to have_many :projects }
@@ -153,25 +154,18 @@ describe Namespace do
     end
   end
 
-  describe '#move_dir' do
+  describe '#move_dir', :request_store do
     let(:namespace) { create(:namespace) }
     let!(:project) { create(:project_empty_repo, namespace: namespace) }
 
-    before do
-      allow(namespace).to receive(:path_changed?).and_return(true)
-    end
-
     it "raises error when directory exists" do
       expect { namespace.move_dir }.to raise_error("namespace directory cannot be moved")
     end
 
     it "moves dir if path changed" do
-      new_path = namespace.full_path + "_new"
+      namespace.update_attributes(path: namespace.full_path + '_new')
 
-      allow(namespace).to receive(:full_path_was).and_return(namespace.full_path)
-      allow(namespace).to receive(:full_path).and_return(new_path)
-      expect(namespace).to receive(:remove_exports!)
-      expect(namespace.move_dir).to be_truthy
+      expect(gitlab_shell.exists?(project.repository_storage_path, "#{namespace.path}/#{project.path}.git")).to be_truthy
     end
 
     context "when any project has container images" do

spec/services/system_note_service_spec.rb

@@ -503,20 +503,6 @@ describe SystemNoteService do
     end
   end
 
-  describe '.cross_reference?' do
-    it 'is truthy when text begins with expected text' do
-      expect(described_class.cross_reference?('mentioned in something')).to be_truthy
-    end
-
-    it 'is truthy when text begins with legacy capitalized expected text' do
-      expect(described_class.cross_reference?('mentioned in something')).to be_truthy
-    end
-
-    it 'is falsey when text does not begin with expected text' do
-      expect(described_class.cross_reference?('this is a note')).to be_falsey
-    end
-  end
-
   describe '.cross_reference_disallowed?' do
     context 'when mentioner is not a MergeRequest' do
       it 'is falsey' do