The observed faulty state transition is probably hard to test,
because we need to hook into internal states to observe them.
Namely this:

    07:30:16 | Build#ruby-2.2 enqueue: created -> pending
    07:30:16 | Pipeline#32    enqueue: created -> pending
    07:30:16 | Build#ruby-2.3 enqueue: created -> pending
    07:30:16 | Build#ruby-2.2     run: pending -> running
    07:30:16 | Pipeline#32        run: pending -> running
    07:30:29 | Build#ruby-2.2    drop: running -> failed
    07:30:29 | Pipeline#32        run: running -> running
    07:30:29 | Build#ruby-2.3     run: pending -> running
    07:30:30 | Pipeline#32        run: running -> running
    07:30:57 | Build#gem:build   skip: created -> skipped
    07:30:57 | Pipeline#32       drop: running -> failed
    07:30:57 | Build#gem:release skip: created -> skipped
    07:30:57 | Pipeline#32       drop:  failed -> failed
    07:30:57 | Build#ruby-2.3    drop: running -> failed
    07:30:57 | Pipeline#32       drop: running -> failed
                                       ^^^ Should be failed -> failed

However, the consequence of this, executing hooks twice would be
easy enough to observe. So we could at least test against this.
Keep in mind that if we ever changed how we execute the hooks
this won't be testing against faulty state transition.

This reverts commit 575dc2b0.

Because pipeline status could be changed for the builds in the next
stages, if we process next stages first, the current build would be
out of synchronized, and would need a reload for that matter.

Alternatively, like what I did in this commit, we could process the
next stages later (by using `after_transition` rather than
`around_transition`), and complete what're doing for the current
build first. This way we don't have to reload because nothing is
out synchronized.

Note that since giving `false` in `after_transition` would halt the
callbacks chain, according to:

https://github.com/state-machines/state_machines-activemodel/blob/v0.4.0/lib/state_machines/integrations/active_model.rb#L426-L429

We'll need to make sure we're not returning false because we don't
intend to interrupt the chain.

This fixes #22010.

After this fix, both pipeline events and build events would only show
up once.

API: Use Search::GlobalService in projects search API

See merge request !6280

Fix API issues sorting

## What does this MR do?

Fix the sorting of issues in the API.

## Are there points in the code the reviewer needs to double check?

Instead of removing the '_at' suffix manually, we could add those versions to the `Sortable` concern instead.

## Why was this MR needed?

There were a couple of bugs:

* The global and project-specific issues endpoints wouldn't sort at all.
* Group sorting would work, but only if you applied two undocumented workarounds:
  * Always pass both `order_by` and `sort` (both are optional, so only one should be needed to change ordering).
  * Instead of passing `created_at` or `updated_at`, you needed to pass `created` or `updated`.

This makes the API implementation match the docs.

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] API support added
- Tests
  - [x] Added for this feature/bug
  - [ ] All builds are passing
- [x] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

## What are the relevant issue numbers?

Closes https://gitlab.com/gitlab-org/gitlab-ee/issues/983.

See merge request !6281

Also streamline the sorting part while we're at it.

That being done, there's currently a duplication between
`GET /projects/search/:query` and `GET /projects?search=:search`
so we might want to keep only the latter for 9.0...
Signed-off-by: Rémy Coutable <remy@rymai.me>

Add notification_settings API calls

## What does this MR do?
It adds API calls to get/update notification settings, globally and at the group/project level.

## Are there points in the code the reviewer needs to double check?
None

## Why was this MR needed?
Notification settings were updated, but related API calls were never created.

## What are the relevant issue numbers?
gitlab-org/gitlab-ce#19359

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [x] API support added
- Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

Closes #19359

See merge request !5632

Pull import sources from Gitlab::ImportSources

## What does this MR do?
`ApplicationSettings.create_from_defaults` defines an array of import sources, but we should be pulling them from `Gitlab::ImportSources.options` since that's what they're validated against.

## What are the relevant issue numbers?

Closes #21814

See merge request !6293

Improve PipelinesFinder spec so that it does not depend on hard-coded database IDs

Fixes failed builds such as https://gitlab.com/gitlab-org/gitlab-ce/builds/3919501

See merge request !6291

Fixes failed builds such as https://gitlab.com/gitlab-org/gitlab-ce/builds/3919501

Enable pipeline events by default

This should probably be included in !5620 but missed.

See merge request !6278

Restrict ldap return attributes

## What does this MR do?

Fixes the CE part of #13821. We really only ever need uid, dn, cn, and mail attributes, and in some cases, even less. This merge request strips the request down to those four attributes by default, and allows the caller to specify others, if needed. 

## Why was this MR needed?

This will improve performance especially in cases where the connection is slow between GitLab and LDAP, or when the LDAP object has lots of attributes we don't care about.

See merge request !6187

Pass the remember_me option into the u2f form and support it while authenticating

## What does this MR do?

Adds remember me support in the u2f authentication, and makes sure the flag gets passed from the login form to the u2f form.

Based on the changes for the same thing done for regular 2fa: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4369

## Why was this MR needed?

The remember me option wasn't working for u2f devices (yubikey)

## What are the relevant issue numbers?

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/18103

See merge request !5918

Archive CHANGELOG entries prior to 8.0 in CHANGELOGS/archive.md

[ci skip]

See merge request !6162

See gitlab-org/release-tools!29

[ci skip]

[ci skip]

Hides merge request button on branches page

## What does this MR do?

 If user does not have the correct permissions, the merge request button is hidden

## What are the relevant issue numbers?

Closes #21805

See merge request !6181

Use NotificationSetting::EMAIL_EVENTS for params

Use gitlab-workhorse 0.8.1

Adds Sentry support in gitlab-workhorse.

See merge request !6283

Avoid conflict with Admin labels when importing GitHub labels

If the GitHub project have duplicated labels from the Admin labels, the importer will use the Admin label.

Fixes #21319

See merge request !6158

This adds (basic) Sentry support.

Restore comments lost when converting CoffeeScript to JavaScript

## What does this MR do?

Restores comments lost in the great fire of !5464.

## Are there points in the code the reviewer needs to double check?

## Why was this MR needed?

#20300

## What are the relevant issue numbers?

 - #20300 (comments)
 - #20098 (ES6)

## Does this MR meet the acceptance criteria?

- [ ] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] API support added
- Tests
  - [ ] Added for this feature/bug
  - [ ] All builds are passing
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
Closes #20300

See merge request !5541

Not sure why I missed this before. I thought I added it.

Minor update on CI docs examples

cc @ayufan @markpundsack 

Commit message: 

> Triggered by gitlab-org/gitlab-ci-yml#1 I've investigated how this is
organized so far and did a minor clean up. In my opinion we should just
keep a list of up to date blog posts in the docs and examples should be
included in gitlab-org/gitlab-ci-yml. This reduces the number of
projects and docs pages one has to look for information.

See merge request !5806

Security and safety improvements for gitlab-workhorse integration

Companion to https://gitlab.com/gitlab-org/gitlab-workhorse/merge_requests/60

- Use a custom content type when sending data to gitlab-workhorse
- Verify (using JWT and a shared secret on disk) that internal API requests came from gitlab-workhorse

This will allow us to build features in gitlab-workhorse that require
more trust, and protect us against programming mistakes in the future.

This is designed so that no action is required for installations from
source. For omnibus-gitlab we need to add code that manages the shared
secret.

See merge request !5907

Fix intermittent spec failures with spec/features/issues/user_uses_slash_commands_spec.rb

This MR solves the intermittent spec failure:

```
 1) Issues > User uses slash commands behaves like issuable record that supports slash commands in its description and notes note on issue with a note changing the issue's title when current user cannot change title of issue does not reopen the issue
     Failure/Error: Gitlab::Routing.url_helpers.namespace_project_url(self.namespace, self)

     ActionController::UrlGenerationError:
       No route matches {:action=>"show", :controller=>"projects", :id=>#<Project id: 1, name: "project135", path: "gitlabhq", description: nil, created_at: "2016-08-31 20:45:48", updated_at: "2016-08-31 20:45:48", creator_id: 2, issues_enabled: true, merge_requests_enabled: true, wiki_enabled: true, namespace_id: 2, snippets_enabled: true, last_activity_at: "2016-08-31 20:45:48", import_url: nil, visibility_level: 20, archived: false, avatar: nil, import_status: "none", repository_size: 0.0, star_count: 0, import_type: nil, import_source: nil, commit_count: 0, import_error: nil, ci_id: nil, builds_enabled: true, shared_runners_enabled: true, runners_token: "_rNZHgsHzaDWGbyL3L2F", build_coverage_regex: nil, build_allow_git_fetch: true, build_timeout: 3600, pending_delete: false, public_builds: true, pushes_since_gc: 0, last_repository_check_failed: nil, last_repository_check_at: nil, container_registry_enabled: true, only_allow_merge_if_build_succeeds: false, has_external_issue_tracker: false, repository_storage: "default", request_access_enabled: true, has_external_wiki: false, lfs_enabled: nil>, :namespace_id=>nil} missing required keys: [:namespace_id]
     Shared Example Group: "issuable record that supports slash commands in its description and notes" called from ./spec/features/issues/user_uses_slash_commands_spec.rb:6
     # ./app/models/project.rb:611:in `web_url'
     # ./app/models/project.rb:975:in `hook_attrs'
     # ./lib/gitlab/data_builder/note.rb:58:in `build_base_data'
     # ./lib/gitlab/data_builder/note.rb:38:in `build'
     # ./app/services/notes/post_process_service.rb:19:in `hook_data'
     # ./app/services/notes/post_process_service.rb:23:in `execute_note_hooks'
     # ./app/services/notes/post_process_service.rb:14:in `execute'
     # ./app/workers/new_note_worker.rb:10:in `perform'
     # ./app/services/notes/create_service.rb:29:in `execute'
     # ./app/controllers/projects/notes_controller.rb:26:in `create'
     # ./lib/gitlab/request_profiler/middleware.rb:15:in `call'
     # ./lib/gitlab/middleware/go.rb:16:in `call'
     # ./lib/gitlab/middleware/static.rb:9:in `call'
```

There were a number of issues:
    
1. Sidekiq async tasks were being run after project namespaces were deleted. Use Sidekiq in fake mode to prevent this from Sidekiq tasks from being enqueued in the first place.
2. Some outstanding Ajax requests (e.g. autocomplete) were still being loaded while `DatabaseCleaner` ran
3. `logout` did not always wait for logout actually to complete
  
Closes #21723

See merge request !6271

Triggered by gitlab-org/gitlab-ci-yml#1 I've investigated how this is
organized so far and did a minor clean up. In my opinion we should just
keep a list of up to date blog posts in the docs and examples should be
included in gitlab-org/gitlab-ci-yml. This reduces the number of
projects and docs pages one has to look for information.

[ci skip]

Refresh todos count cache when an Issue/MR is deleted

Fixes #21714

See merge request !6161

* Moves splitting of  `ps` result to helper.
* Users a regexp to extract the different parts

The result of `ps` at least depends on the _LANG_ and the distance of
_start_ to _now_ (start time then the day + hour then month+day)
I've tested with _LANG=en_US.UTF-8_ and _LANG=de_CH.UTF-8_

Closes #20632

See merge request !6245