This change simplifies a Pipeline processing by introducing a special new status: created.
This status is used for all builds that are created for a pipeline.
We are then processing next stages and queueing some of the builds (created -> pending) or skipping them (created -> skipped).
This makes it possible to simplify and solve a few ordering problems with how previously builds were scheduled.
This also allows us to visualise a full pipeline (with created builds).

This also removes an after_touch used for updating a pipeline state parameters.
Right now in various places we explicitly call a reload_status! on pipeline to force it to be updated and saved.

Remove trigram indexes for "ci_runners"

## What does this MR do?

This MR removes two trigram indexes from the `ci_runners` table.

## Are there points in the code the reviewer needs to double check?

No.

## Why was this MR needed?

These indexes are only very rarely used while slowing down any update. See 4baee5d3a2e9e3b49a991db89e335072b911e64c for more information.

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] ~~[Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)~~
- [x] ~~API support added~~
- Tests
  - [x] ~~Added for this feature/bug~~
  - [ ] All builds are passing
- [x ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5755

Fix branches page dropdown sort initial state

## What does this MR do?
Add 'Name' as the initial state for the branches dropdown filter

## Are there points in the code the reviewer needs to double check?
Shouldn't be

## Why was this MR needed?
Prevent dropdown from being rendered without an initial state

## What are the relevant issue numbers?
Closes #20753 

## Screenshots (if relevant)
Before:
![Screen_Shot_2016-08-10_at_2.26.59_PM](/uploads/e13725c90d1e870e0b08c3bff9234c4e/Screen_Shot_2016-08-10_at_2.26.59_PM.png)

After:
![Screen_Shot_2016-08-10_at_2.26.45_PM](/uploads/cd4957902ee1ebb74ac6e93b18c8ed02/Screen_Shot_2016-08-10_at_2.26.45_PM.png)

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5761

Convert shortcuts image into markdown

## What does this MR do?
Recreates shortcuts page into markdown

## Are there points in the code the reviewer needs to double check?
Double check all the shortcuts are correct

## Why was this MR needed?
Improve maintainability of this page

## What are the relevant issue numbers?
Closes #20160 

See merge request !5452

Update container_registry.md

## What does this MR do?

Add a warning about what ports to choose for the external registry URL

## Are there points in the code the reviewer needs to double check?

## Why was this MR needed?

It would have saved me about 3 hours if this line was in the document.

## What are the relevant issue numbers?

## Screenshots (if relevant)

## Does this MR meet the acceptance criteria?

It's just a one-line documentation addition.

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [x] API support added
- Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5238

Fix awardable button mutuality loading spinners

## What does this MR do?
Removes the showing and hiding of loading spinners when an awardable's opposite button is selected. 

## Are there points in the code the reviewer needs to double check?
Shouldn't be

## Why was this MR needed?
Makes the UI more consistent. Reduces confusion of why another button is showing the loading spinner when it was not clicked on.

## What are the relevant issue numbers?
Closes #18380 

## Screenshots (if relevant)
Before:
![loading](/uploads/d4c4e565fa58fbbf3db6ed8e28f9abb0/loading.gif)

After:
![CcwRYZcUVm](/uploads/b93fd04853f43a6c70de3cefb20d08a1/CcwRYZcUVm.gif)

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5745

Remove two simple instances of inline JavaScript.

Not very complicated, they're moved to the dispatcher.

See merge request !5513

Avoid commit lookup on diff_helper

See merge request !5756

Store OTP secret key in secrets.yml

## What does this MR do?

Migrate the value of `.secret` to `config/secrets.yml` if present, so that `.secret` can be rotated without preventing all users with 2FA from logging in. (On a clean setup, generate different keys for each.)

## Are there points in the code the reviewer needs to double check?

I'm not sure we actually need `.secret` at all after this, but it seems safer not to touch it.

## Why was this MR needed?

We have some DB encryption keys in `config/secrets.yml`, and one in `.secret`. They should all be in the same place.

## What are the relevant issue numbers?

#3963, which isn't closed until I make the relevant changes in Omnibus too.

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- ~~API support added~~
- Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5274

Clean up project destruction

Instead of redirecting from the project service to the service and back to the model, put all destruction code in the service. Also removes a possible source of failure where run_after_commit may not destroy the project.

See merge request !5695

Use badge image template instead of using separate images

## What does this MR do?

Makes it possible to use template for badge instead of having multiple files.

## Are there points in the code the reviewer needs to double check?

We also have a deprecated badge in `controllers/ci/projects_controller.rb`.  We decided to leave it until 9.0, so we still have images in `public/ci/` until 9.0.

## Why was this MR needed?

We are going to implement build coverage badge, and we do not want to store 101 SVG images for each percentage value.

## What are the relevant issue numbers?

#3714

## Screenshots (if relevant)

![new_build_badge](/uploads/f1d4ed5e34278eb01f48994b5b0579f1/new_build_badge.png)

## Does this MR meet the acceptance criteria?

- [ ] ~~[CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added~~ (refactoring)
- [ ] ~~[Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)~~
- [ ] ~~API support added~~
- Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [ ] ~~[Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)~~ (refactoring)

See merge request !5520

Signed-off-by: Rémy Coutable <remy@rymai.me>

Instead of redirecting from the project service to the service and back to the model,
put all destruction code in the service. Also removes a possible source of failure
where run_after_commit may not destroy the project.

Mention add_column_with_default in downtime guide

## What does this MR do?

This extends the "What causes downtime" guide with a mention of the `add_column_with_default` migration helper.

## Are there points in the code the reviewer needs to double check?

Spelling, grammar, etc.

## Why was this MR needed?

Currently the guide may lead one to believe it's not possible at all to add a column with a default value.

## Does this MR meet the acceptance criteria?

- [x] ~~[CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added~~
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- Tests
  - [ ] All builds are passing
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5719

Fix escaped angle bracket's in integration documentation.

There are several escaped angle brackets in our integration docs.  While these
render fine within GitLab, they are broken when rendered on doc.gitlab.com because
pandoc does not escape them correctly.

You can see the problem here: http://doc.gitlab.com/ce/integration/github.html

Note that the strings `<Organization>` and `<Your Name>` are being interpreted as html tags.

It looks like doc.gitlab.com is using pandoc: https://gitlab.com/gitlab-com/doc-gitlab-com/blob/master/generate.rb#L64

You can verify these changes by running something like this and then inspecting `test.html`:

    pandoc --from markdown_github-hard_line_breaks -o test.html doc/integration/twitter.md

You can also verify that GitLab continues to render the docs correctly by checking, for example, http://localhost:3000/help/integration/twitter.md


See merge request !4128

Fix filter label tooltip HTML rendering

## What does this MR do?
Tooltips on labels used in a filter are now rendered as HTML

## Are there points in the code the reviewer needs to double check?
Shouldn't be

## Why was this MR needed?
Fix existing bug (See screenshot below for reference)

## What are the relevant issue numbers?
Closes #20592 

## Screenshots (if relevant)
Before:
![Screen_Shot_2016-08-04_at_10.27.59_AM__2_](/uploads/7ca37d4116f15a2d22ad16fbbe682f1f/Screen_Shot_2016-08-04_at_10.27.59_AM__2_.png)

After:
![Screen_Shot_2016-08-04_at_10.24.37_AM__2_](/uploads/7fce6c8a93d293767dacbcd682831f86/Screen_Shot_2016-08-04_at_10.24.37_AM__2_.png)

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5671

Used cached value of project count to reduce DB load

Following !5746, backport a change from https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/637/diffs#diff-2.

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5754

Update UI Guide with SVG guidelines

This addition to the guide is to provide some guidelines to UX designers when exporting SVGs. Please let me know if anything is unclear or if you any improvements so we can document it clearly for everyone.

cc / @hazelyang @cperessini @dimitrieh @connorshea @annabeldunstone @dzaporozhets 

## What are the relevant issue numbers?
https://gitlab.com/gitlab-org/gitlab-ee/issues/872

See merge request !5748

Small refactor of doc/development/README.md



See merge request !5757

Add an initial .mailmap

## What does this MR do?

Add an intiail ".mailmap" file to better map users names and email adresses for `git shortlog` and `git check-mailmap` to work better.

## Are there points in the code the reviewer needs to double check?

The reviewer can try `git shortlog -nse` with and without the ".mailmap" to see the difference.

## Why was this MR needed?

It starts to make `git shortlog` and `git check-mailmap` useful.

## What are the relevant issue numbers?

I don't know of any. I searched for "mailmap" and found nothing.

## Screenshots (if relevant)

## Does this MR meet the acceptance criteria?

I can add an entry into the CHANGELOG but this is for developers only so I am not sure it is really needed.

- [ ] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] API support added
- Tests
  - [ ] Added for this feature/bug
  - [ ] All builds are passing
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5583

- Fix wrong doc_styleguide URL
- Clarify migrations refer to SQL
- Capitalize CSS

These indexes are only used when you search for runners in the admin
interface. This operation is so rarely used that it does not make sense
to slow down every update in order to update the GIN trigram indexes.

Removing these indexes should speed up queries such as those used for
updating the last contact time of CI runners. Locally the timings of
this query were reduced from ~50 ms to ~25 ms:

    UPDATE ci_runners SET updated_at = now(), contacted_at = now();

Signed-off-by: Rémy Coutable <remy@rymai.me>

Use FA GitLab Icon for import project button

Before:

![Screen_Shot_2016-08-08_at_09.43.47](/uploads/e419dc0d229ff848294cf277ffbb02f9/Screen_Shot_2016-08-08_at_09.43.47.png)

After:

![Screen_Shot_2016-08-08_at_09.43.09](/uploads/3a77c5e98385bbc1c7139300eeca5866/Screen_Shot_2016-08-08_at_09.43.09.png)

See merge request !5702

[ci skip]

Link to doc root, not readme of GDK

I hope this makes navigation between gitlab-ce developer docs and GDK docs more smooth.

See merge request !5716

Use long options for curl examples in documentation

## What does this MR do?

Use long options (e.g. `--header` instead of `-H`) for curl examples in documentation.

## Why was this MR needed?

Short options are less readable.

## What are the relevant issue numbers?

https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5465#note_13603730


See merge request !5703

Clarify the features for generating default label sets

## What does this MR do?

* Split out the information regarding the label set that GitLab can generate
* Add information for the labels view in the Admin panel

## Are there points in the code the reviewer needs to double check?

* Please ensure that the newly added section relating to the Admin area seems relevant
* Please verify adherence to the documentation styleguide

## Why was this MR needed?

Documentation regarding default label sets was missing

## What are the relevant issue numbers?
Closes #1865

See merge request !5736

Rename `run` task helper method to prevent conflict with StateMachine

This prevents the following message from appearing whenever running a
Rake task:

    Instance method "run" is already defined in Object, use generic
    helper instead or set StateMachines::Machine.ignore_method_conflicts
    = true.

See merge request !5750

Add a method in Project to return a cached value of total count of projects

This is in preparation to address the DB load caused by the counting
in gitlab-com/infrastructure#303.

See merge request !5746

This prevents the following message from appearing whenever running a
Rake task:

    Instance method "run" is already defined in Object, use generic
    helper instead or set StateMachines::Machine.ignore_method_conflicts
    = true.

Check out GH pull requests locally where the source/target branch had been deleted

## What does this MR do?

Check out GitHub pull requests where source/target branches that had been deleted locally rather than temporarily restoring them on GitHub using their References API. This helps us to not get rate limited, and allow us to import cross-repository pull requests (those from forks).

## What are the relevant issue numbers?

Fixes #15528

Fixes #17766

Fixes #19310

Fixes #19439

Fixes #19998

Fixes #20153

Fixes #20552

Fixes https://gitlab.com/gitlab-com/support-forum/issues/801

#20385

## Screenshots (if relevant)

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [X] ~~API support added~~
- Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5630

# Conflicts:
#	lib/gitlab/redis.rb