Update gitlab-shell to v3.3.3

Finishing touch for https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5542

![](https://gitlab.com/gitlab-org/gitlab-shell/uploads/bd0d4864f7bd864d38bfb6798fdd4ea3/image.png)

See merge request !5780

Upgrade Rails to 4.2.7.1 for security fixes.

Upgrades Rails from 4.2.7 to 4.2.7.1 for security fixes.

For more information: http://weblog.rubyonrails.org/2016/8/11/Rails-5-0-0-1-4-2-7-2-and-3-2-22-3-have-been-released/

This should be backported to all currently-supported releases.

See merge request !5781

More information available at: http://weblog.rubyonrails.org/2016/8/11/Rails-5-0-0-1-4-2-7-2-and-3-2-22-3-have-been-released/

Merge request link after pushing

## What does this MR do?
Add API to generate new merge request urls if it is a new branch. The API will be called by gitlab-shell 

## What are the relevant issue numbers?

#18266 

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] API support added
- Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5542

Use Ruby 2.3.1

## What does this MR do?

This WIP MR will eventually introduce 2.3 as default language for GitLab CE/EE

## Why was this MR needed?

@connorshea wants to be prepared for Rails 5.0, so we need Ruby > 2.2, so why not go 2.3 already?

## What are the relevant issue numbers?

Closes #12507 

## TODO

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [x] All builds are passing
- [ ] [Omnibus uses 2.3.1](https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/847)

See merge request !4948

Remove need for schedules

Solves https://gitlab.com/gitlab-org/gitlab-ce/issues/15156

\cc @pmachle ,  \cc @dusek

See merge request !3685

Ruby 2.1.8 is used only on master for now, as this would give
confidance in case this commit has to be reverted.

Show member roles to all users on members page

Closes #20614 and https://gitlab.com/gitlab-org/gitlab-ee/issues/850

See merge request !5776

DRY code + fix rubocop

Add more test cases

Append to changelog

DRY changes list

find_url service for merge_requests

use GET for getting merge request links

remove files

rename to get_url_service

reduce loop

add test case for cross project

refactor tiny thing

update changelog

Rename `markdown_preview` routes to `preview_markdown`

_Originally opened at !3700 by @cbartz._

Closes #15000.

See merge request !5774

Remove Grack::Auth: part 2 (LFS)

Deprecate Grack::Auth and handle LFS in Rails controllers under the Project namespace.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/14501

See merge request !5369

Refactor pipeline creation service

## What does this MR do?
This refactors GitLab CI build processing: all builds for pipeline are pre-created when a pipeline object is created. 
The builds are created with a new introduced status `created`.
The builds are then automatically promoted to `pending` when a previous stage do succeed.
This significantly simplifies pipeline processing code solving a lot of problems of lazily initialisation of previous approach (builds were created on-demand).

## Why was this MR needed?
The previous mechanism had a lot of flows (shown in related issues) in how it work, but also in code design. Removing cross model-service-library dependencies.

The current approach moves a build creation to single place `CreatePipelineService` and removes a dynamic dependency on `config_processor` significantly simplifying a build creation and pipeline processing. Pipeline processing is implemented in `ProcessPipelineService`.

This also allows to easily extend GitLab with Manual Actions which is part of 8.10 direction issue.

## Migration problem
~~This MR removes the a on-demand creation of builds in pipelines.
 Pipelines that are running and are in mid-stage (some stages started, but not all) will not be fully evaluated after application restart. 
This happens, because the code responsible for on-demand creation is removed. 
There's no easy way to migrate existing pipelines, other than doing offline migration and putting pipeline processing in migration code (which seems to be a really bad idea).~~

To support old pipelines I added a lazy initialization of builds if none is found.

## What are the relevant issue numbers?
Fixes: https://gitlab.com/gitlab-org/gitlab-ce/issues/12839
Solves: https://gitlab.com/gitlab-org/gitlab-ce/issues/18644 https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/289
Allows to easily implement: https://gitlab.com/gitlab-org/gitlab-ce/issues/17010

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [x] API support added
- Tests
  - [x] Added for this feature/bug
  - [ ] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)


See merge request !5295

This change simplifies a Pipeline processing by introducing a special new status: created.
This status is used for all builds that are created for a pipeline.
We are then processing next stages and queueing some of the builds (created -> pending) or skipping them (created -> skipped).
This makes it possible to simplify and solve a few ordering problems with how previously builds were scheduled.
This also allows us to visualise a full pipeline (with created builds).

This also removes an after_touch used for updating a pipeline state parameters.
Right now in various places we explicitly call a reload_status! on pipeline to force it to be updated and saved.

Fix front-end for branches that happen to contain urlencoding escape characters (e.g. %)

_Originally opened at !3574 by @ewiltshi._

- - -

Adding a branch with a name like "foo%20bar" (via command-line) and setting it to the project's default branch causes the project "show" page to 404 for that project. "assign_ref_vars" unescapes the branch name ("foo%20bar" ==> "foo bar"), making GitLab look for a non-existent branch.
This MR adds logic to skip the URL unescaping step in "assign_ref_vars".

Fixes #10772, #14992, #15304.

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
  - [x] Added for this feature/bug
  - [ ] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5770

Use `File::exist?` instead of `File::exists?`

Each `File::exists?` replaced to `File::exist?`.

Since version ruby-2.2.0, method `File::exists?` is deprecated.

See merge request !5747

Remove various redundant indexes

## What does this MR do?

This MR removes various redundant and unused indexes. See c3d5c536d9c77dbe183744cb19f2cdab7ee606fd for more information.

## Are there points in the code the reviewer needs to double check?

No.

## Why was this MR needed?

Unused indexes are a waste of storage space and slow down database writes. If they are ever needed again we can re-create them. There's no point in hoarding indexes because you _may_ need them in the future.

## What are the relevant issue numbers?

#20767

## Screenshots (if relevant)

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
  - [ ] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5759

Add group members API endpoints to request access and approve requests

## What does this MR do?

Add group members API endpoints to request access and approve requests.

## Are there points in the code the reviewer needs to double check?

I chose to factorize the group/project members API as well as the new group/project access requests API.

I initially also created new services to centralize the permission checks related to actions on members, but this was out of scope and this MR is already quite big so I opened a temp MR at !5566 based on this one.

## Why was this MR needed?

To finish the "request access" feature.

## What are the relevant issue numbers?

Closes #18583.

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [x] API support added
- Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !4833

Fixing scope issue in GitAccess.

This MR fixes a scoping issue around the GitAccessStatus.new call - it needs to be scoped to the Gitlab otherwise it can end up being confused with the GitAccessStatus class in gitlab-shell, which has a 3-element constructor.

Although not strictly necessary for community GitLab, it is more correct, and fixes an issue we (Perforce) found due to our engine overrides.

This change should have any affect on community GitLab.

See merge request !5483

Replace the tinder gem by bare HTTP requests

## What does this MR do?

It removes the `tinder` gem (used to talk to the Campfire API) and replaces its use by bare HTTP requests.

## Are there points in the code the reviewer needs to double check?

N/A.

## Why was this MR needed?

To simplify the potential dependency hell discussed in #13333.  
The initial issue was about updating `http_parser.rb` which was required by `twitter-stream` which was required by `tinder`.  
Those 3 gems are not needed anymore.  

## What are the relevant issue numbers?

#13333

## Screenshots (if relevant)

N/A.

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
Closes #13333

See merge request !5758

Signed-off-by: Rémy Coutable <remy@rymai.me>

One can see which indexes are used in PostgreSQL by running the
following query:

    SELECT relname as table_name, indexrelname as index_name, idx_scan, idx_tup_read, idx_tup_fetch, pg_size_pretty(pg_relation_size(indexrelname::regclass))
    FROM pg_stat_all_indexes
    WHERE schemaname = 'public'
    AND "idx_scan" = 0
    ORDER BY pg_relation_size(indexrelname::regclass) desc;

Using this query I built a list of indexes that could be potentially
removed. After checking every single one by hand to make sure they
really aren't used I only found 1 index that _would_ be used. This was a
GitLab GEO index (EE) specific that's currently not used simply because
the table is empty.

Apart from this one index all indexes could be removed. The migration
also takes care of 6 composite indexes that can be replaced with a
single column index, which in most cases was already present.

For more information see gitlab-org/gitlab-ce#20767.

Since version ruby-2.2.0, method `File::exists?` is deprecated.

Update/Improve OAuth2 Client documentation

## What does this MR do?
Improves the OAuth2 client documentation to include security hints to help people create secure clients.

## Are there points in the code the reviewer needs to double check?

## Why was this MR needed?
I noticed the docs didn't mention some important points and steps, this cleans that up.

## What are the relevant issue numbers?

## Screenshots (if relevant)

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] API support added
- Tests
  - [ ] Added for this feature/bug
  - [ ] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5665

Signed-off-by: Rémy Coutable <remy@rymai.me>

Remove trigram indexes for "ci_runners"

## What does this MR do?

This MR removes two trigram indexes from the `ci_runners` table.

## Are there points in the code the reviewer needs to double check?

No.

## Why was this MR needed?

These indexes are only very rarely used while slowing down any update. See 4baee5d3a2e9e3b49a991db89e335072b911e64c for more information.

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] ~~[Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)~~
- [x] ~~API support added~~
- Tests
  - [x] ~~Added for this feature/bug~~
  - [ ] All builds are passing
- [x ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5755

Fix branches page dropdown sort initial state

## What does this MR do?
Add 'Name' as the initial state for the branches dropdown filter

## Are there points in the code the reviewer needs to double check?
Shouldn't be

## Why was this MR needed?
Prevent dropdown from being rendered without an initial state

## What are the relevant issue numbers?
Closes #20753 

## Screenshots (if relevant)
Before:
![Screen_Shot_2016-08-10_at_2.26.59_PM](/uploads/e13725c90d1e870e0b08c3bff9234c4e/Screen_Shot_2016-08-10_at_2.26.59_PM.png)

After:
![Screen_Shot_2016-08-10_at_2.26.45_PM](/uploads/cd4957902ee1ebb74ac6e93b18c8ed02/Screen_Shot_2016-08-10_at_2.26.45_PM.png)

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5761

Convert shortcuts image into markdown

## What does this MR do?
Recreates shortcuts page into markdown

## Are there points in the code the reviewer needs to double check?
Double check all the shortcuts are correct

## Why was this MR needed?
Improve maintainability of this page

## What are the relevant issue numbers?
Closes #20160 

See merge request !5452

Update container_registry.md

## What does this MR do?

Add a warning about what ports to choose for the external registry URL

## Are there points in the code the reviewer needs to double check?

## Why was this MR needed?

It would have saved me about 3 hours if this line was in the document.

## What are the relevant issue numbers?

## Screenshots (if relevant)

## Does this MR meet the acceptance criteria?

It's just a one-line documentation addition.

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [x] API support added
- Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5238

Fix awardable button mutuality loading spinners

## What does this MR do?
Removes the showing and hiding of loading spinners when an awardable's opposite button is selected. 

## Are there points in the code the reviewer needs to double check?
Shouldn't be

## Why was this MR needed?
Makes the UI more consistent. Reduces confusion of why another button is showing the loading spinner when it was not clicked on.

## What are the relevant issue numbers?
Closes #18380 

## Screenshots (if relevant)
Before:
![loading](/uploads/d4c4e565fa58fbbf3db6ed8e28f9abb0/loading.gif)

After:
![CcwRYZcUVm](/uploads/b93fd04853f43a6c70de3cefb20d08a1/CcwRYZcUVm.gif)

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5745

Remove two simple instances of inline JavaScript.

Not very complicated, they're moved to the dispatcher.

See merge request !5513