Allow customising of queries used for `update_column_in_batches`

This MR makes two changes to `add_column_with_default` and `update_column_in_batches`:

1. `add_column_with_default` no longer wraps the entire set of updates in a single transaction, preventing any locks from sticking around for the duration of the entire transaction
2. `update_column_in_batches` now takes a block which can be used to customise the queries. This uses Arel as messing with raw SQL strings is a total pain

In !4381 there's a need for updating existing rows/columns in a table in batches using a custom `WHERE` condition. Without the changes in this MR this would not be possible.

See merge request !4680

Fix permission checks in member row (backport from gitlab-org/gitlab-ee!460)

## What does this MR do?

It improves the check we use to display or not the members' access and controls in the members list.

## Are there points in the code the reviewer needs to double check?

No, I replaced an helper with just a permission check so I think it's a better solution.

## Why was this MR needed?

There were a spec failure in gitlab-org/gitlab-ee!460 because of the refactor done in the "request access" MR.

## What are the relevant issue numbers?

None.

## Does this MR meet the acceptance criteria?

- No CHANGELOG needed
- [x] Tests
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !4670

Added API endpoint for Sidekiq Metrics.

## What does this MR do?

It adds an API endpoint to gather metrics about Sidekiq, it's jobs, queues, and processes.

## Why was this MR needed?

There was no API endpoint for Sidekiq information.

## What are the relevant issue numbers?

Fixes #7171

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [x] API support added
- [x] Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing

See merge request !4653

Fixed issue with user calendar colors

## What does this MR do?

Fixes an issue with the colors added onto the user contrib calendar. The calendar was change recently to remove a library which affected how the colors where generated.

## What are the relevant issue numbers?

Closes #18505 

## Screenshots (if relevant)

![Screen_Shot_2016-06-13_at_09.33.10](/uploads/f17451d2eb805e9d75d859cd1691e0bc/Screen_Shot_2016-06-13_at_09.33.10.png)

See merge request !4614

Add sorting dropdown to tag page

## What does this MR do?

Adds a sorting dropdown to the tags page just like the one on the branches page.

## Are there points in the code the reviewer needs to double check?

No

## Why was this MR needed?

Clients were asking for this

## What are the relevant issue numbers?

Closes #14918

## Screenshots (if relevant)

![Captura_de_pantalla_2016-06-01_a_las_4.07.58_p.m.](/uploads/4530683ddd91d3bdbdce77748fe63f87/Captura_de_pantalla_2016-06-01_a_las_4.07.58_p.m..png)

See merge request !4423

Use gitlab-git 10.2.0

Closes #18741

See merge request !4722

Banzai::Filter::UploadLinkFilter use XPath

See merge request !4703

Fixed ordering in Project.find_with_namespace

This MR fixes the ordering of `Project.find_with_namespace` to ensure that it returns rows that match literally first.

Closes #18603

See merge request !4682

[ci skip]

Remove JiraIssue model and replace references with ExternalIssue

This MR Removes unused JiraIssue class and replaces references with ExternalIssue

Closes #18203 

See merge request !4659

Add GFM autocomplete for labels

Closes #14637 

See merge request !4013

Added title attribute to entries in tree view

## What does this MR do?

Adds a title attribute to entries in the tree view. Moe useful when files have long names.

## What are the relevant issue numbers?

Closes #18353 


See merge request !4709

Fix project header alignment media query bug

## What does this MR do?
Fixes media query bug on project header

## Screenshots (if relevant)
![Screen_Shot_2016-06-15_at_3.22.20_PM](/uploads/3d84f8ffb88c316f317048e15ffea4f3/Screen_Shot_2016-06-15_at_3.22.20_PM.png)

See merge request !4689

Hide the Todo button in the collapsed issuable sidebar.

## What does this MR do?

Fixes a bug where the "Add Todo" button was displayed in the collapsed sidebar.

## Are there points in the code the reviewer needs to double check?

This shouldn't remove any other elements from the collapsed sidebar.

## What are the relevant issue numbers?

None

## Screenshots (if relevant)

Before:

![Screen_Shot_2016-06-15_at_11.41.46_AM](/uploads/27a1ace58cf8c44ad96accd3d0b32f11/Screen_Shot_2016-06-15_at_11.41.46_AM.png)

After:

![Screen_Shot_2016-06-15_at_11.40.53_AM](/uploads/91a8585680fc482682d0efafdba4d599/Screen_Shot_2016-06-15_at_11.40.53_AM.png)

See merge request !4686

Add bottom padding for merge request command line text

Because without bottom padding text is too close to the UI tabs - hard to navigate



See merge request !4681

Fixed alignment of download dropdown

## What does this MR do?

Correctly aligns the download dropdown on merge requests

## Screenshots (if relevant)

![Screen_Shot_2016-06-14_at_14.47.06](/uploads/9381f25110e5573e56aa5c3f46786df5/Screen_Shot_2016-06-14_at_14.47.06.png)

See merge request !4646

Fixed last push event banner not being in container

## What does this MR do?

Adds the last push event content into a container with the correct class.

## What are the relevant issue numbers?

Closes #18567 

## Screenshots (if relevant)

![Screen_Shot_2016-06-14_at_13.17.41](/uploads/4afa3d2c73e59c5ffbd53869e49baa92/Screen_Shot_2016-06-14_at_13.17.41.png)

See merge request !4644

Fair usage of Shared Runners

## What does this MR do?

Introduces a fair usage scheduler for shared runners.

It tries to assign builds to shared runner from projects that have the lowest number of builds currently running on shared runners.

**Example 1**:
```
We have following builds in queue:
build 1 for project 1
build 2 for project 1
build 3 for project 1
build 4 for project 2
build 5 for project 2
build 6 for project 3

With the new algorithm we will assign builds in following order:
- We choose build 1, because project 1 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We choose build 4, because project 2 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We choose build 6, because project 3 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We choose build 2, because project 1 as other it runs 1 build,
- We choose build 5, because project 2 runs 1 build, where project 1 runs 2 builds now,
- We choose build 3, because project 1 and runs 2 builds.
```

 
**Example 2**:
```
We have following builds in queue:
build 1 for project 1
build 2 for project 1
build 3 for project 1
build 4 for project 2
build 5 for project 2
build 6 for project 3

With the new algorithm we will assign builds in following order:
- We choose build 1, because project 1 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We finish build 1,
- We choose build 2, because project 1 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We choose build 4, because project 2 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We finish build 4,
- We choose build 5, because project 2 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We choose build 6, because project 3 doesn't run currently any builds,
- We choose build 3, because project 1, 2 and 3 runs exactly one build now,
```

## Why was this MR needed?

Currently, we are scheduling builds using FIFO. This is catastrophic if there are projects that create a 100-300 jobs, this basically eats most of available shared runners.

## Performance

All this logic is implemented with the help of SQL queries, because this is the fastest way to process 1k-2k pending builds in queue.
It's not the fastest SQL query, because it sorts based on number of running_builds, and this forces to calculate a number of running builds for all dependent projects. However, since we have one/two shared runners that asks every few seconds for builds this should have minimal impact on DB performance.

```
 explain analyze SELECT "ci_builds".* FROM "ci_builds" JOIN (SELECT "ci_builds"."gl_project_id", count(case when status = 'running' AND runner_id = (SELECT "ci_runners"."id" FROM "ci_runners" WHERE "ci_runners"."is_shared" = 't') then 1 end) as running_builds FROM "ci_builds" INNER JOIN "projects" ON "projects"."id" = "ci_builds"."gl_project_id" AND "projects"."pending_delete" = 'f' WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."status" IN ('running', 'pending') AND "projects"."builds_enabled" = 't' AND "projects"."shared_runners_enabled" = 't' GROUP BY "ci_builds"."gl_project_id") AS projects ON ci_builds.gl_project_id=projects.gl_project_id WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."status" = 'pending' AND "ci_builds"."runner_id" IS NULL  ORDER BY projects.running_builds ASC, ci_builds.id ASC;
                                                                                  QUERY PLAN                                                                        
           
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------
 Sort  (cost=64777.28..64777.29 rows=1 width=1010) (actual time=301.794..302.535 rows=1537 loops=1)
   Sort Key: (count(CASE WHEN (((public.ci_builds.status)::text = 'running'::text) AND (public.ci_builds.runner_id = $0)) THEN 1 ELSE NULL::integer END)), public.ci
_builds.id
   Sort Method: quicksort  Memory: 1423kB
   ->  Nested Loop  (cost=63279.78..64777.27 rows=1 width=1010) (actual time=66.384..298.724 rows=1537 loops=1)
         ->  HashAggregate  (cost=63177.15..63177.30 rows=15 width=15) (actual time=65.641..65.851 rows=187 loops=1)
               InitPlan 1 (returns $0)
                 ->  Seq Scan on ci_runners  (cost=0.00..26963.66 rows=1 width=4) (actual time=1.145..34.381 rows=1 loops=1)
                       Filter: is_shared
                       Rows Removed by Filter: 6965
               ->  Nested Loop  (cost=0.00..36186.34 rows=2715 width=15) (actual time=0.065..29.717 rows=1710 loops=1)
                     ->  Index Scan using index_ci_builds_on_status on ci_builds  (cost=0.00..8913.95 rows=3577 width=15) (actual time=0.051..12.012 rows=2583 loops
=1)
                           Index Cond: ((status)::text = ANY ('{running,pending}'::text[]))
                           Filter: ((type)::text = 'Ci::Build'::text)
                           Rows Removed by Filter: 1219
                     ->  Index Scan using projects_pkey on projects  (cost=0.00..7.61 rows=1 width=4) (actual time=0.003..0.004 rows=1 loops=2583)
                           Index Cond: (id = public.ci_builds.gl_project_id)
                           Filter: ((NOT pending_delete) AND builds_enabled AND shared_runners_enabled)
                           Rows Removed by Filter: 0
         ->  Bitmap Heap Scan on ci_builds  (cost=102.63..106.64 rows=1 width=1002) (actual time=1.216..1.231 rows=8 loops=187)
               Recheck Cond: ((gl_project_id = public.ci_builds.gl_project_id) AND ((status)::text = 'pending'::text))
               Filter: ((runner_id IS NULL) AND ((type)::text = 'Ci::Build'::text))
               ->  BitmapAnd  (cost=102.63..102.63 rows=1 width=0) (actual time=1.201..1.201 rows=0 loops=187)
                     ->  Bitmap Index Scan on index_ci_builds_on_gl_project_id  (cost=0.00..10.52 rows=241 width=0) (actual time=0.406..0.406 rows=1944 loops=187)
                           Index Cond: (gl_project_id = public.ci_builds.gl_project_id)
                     ->  Bitmap Index Scan on index_ci_builds_on_status  (cost=0.00..91.78 rows=3089 width=0) (actual time=0.652..0.652 rows=3362 loops=187)
                           Index Cond: ((status)::text = 'pending'::text)
 Total runtime: 303.832 ms
```

## Specific runners

It doesn't affect the specific runners which still serve builds FIFO.

@stanhu @markpundsack @yorickpeterse What do you think?


See merge request !4634

Shows build scroll buttons after build is complete

## What does this MR do?

Shows the build scroll buttons whenever there is a build trace. Previously they were only shown when the build was active.

## What are the relevant issue numbers?

Closes #18515 

## Screenshots (if relevant)

![Screen_Shot_2016-06-13_at_11.42.42](/uploads/6e0577f46a0cc7b4f2f13b17c85d97da/Screen_Shot_2016-06-13_at_11.42.42.png)

See merge request !4621

Fix emoji block selector.

Fixes #18465 and #18577

See merge request !4604

Fixed issue when opening a highlighted line diff

## What does this MR do?

With the new project nav being fixed, the diff page is scrolling the highlighted under the nav meaning you cant see what is highlighted. This corrects that by added the height of the new project nav into the offset.

See merge request !4597

Added source branch text to dropdown toggle

## What does this MR do?

Previously, the dropdown toggle would default to "Select source branch", this changes that so that it defaults to the branch name and if that doesn't exist, it defaults to "Select source branch"

## Screenshots (if relevant)

![Screen_Shot_2016-06-07_at_12.20.10](/uploads/40d31e21297a99300050cf9f23bd7371/Screen_Shot_2016-06-07_at_12.20.10.png)

See merge request !4504

Reduce SQL query counts in IssueReferenceFilter

## What does this MR do?

This MR adds a preparation phase for reference filters that allows them to prepare/create data structures used while iterating over HTML nodes. In this particular case the preparation phase is used for issue references to greatly cut down the amount of queries executed to get projects/issues for Markdown references.

## Are there points in the code the reviewer needs to double check?

No.

## Why was this MR needed?

Rendering Markdown containing issue references would run at most two queries for every issue reference: one to get the project and one to get the issue from said project. When rendering Markdown with lots of issue references this would result in _a lot_ of queries being executed.

## What are the relevant issue numbers?

#18042

See merge request !4410

Implements TemplateDropdown class to create custom template dropdowns

## What does this MR do?
Refactorize template dropdowns. This MR creates a base TemplateSelector class so it can be reused for multiple types of templates.

## Does this MR meet the acceptance criteria?

- [ ] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] API support added
- [ ] Tests
  - [ ] Added for this feature/bug
  - [x] All builds are passing
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !4697

Use gitlab_git 10.1.4

Towards https://gitlab.com/gitlab-org/gitlab-ce/issues/18690,
following https://gitlab.com/gitlab-org/gitlab_git/merge_requests/87 .

See merge request !4706

Fix images in emails

Closes #18474. The confirmation email is actually the only one that uses the `image_tag` helper at the moment, but this will work for all of them:
```shell
$ ag image_tag -G mailer
app/views/layouts/devise_mailer.html.haml
13:              = image_tag('mailers/gitlab_header_logo.png', id: 'logo', alt: 'GitLab Wordmark')
24:                  = image_tag('mailers/gitlab_tanuki_2x.png', alt: 'GitLab Logo')
```

See merge request !4705

Banzai::Filter::ExternalLinkFilter use XPath

See merge request !4702

Update migration_style_guide.md with new details

See merge request !4693

Lighten each logo path color instead of white

## What does this MR do?
Changes highlight/loading colors of tanuki from white to a lighter version of each path color

## Why was this MR needed?
Since the logo is no longer on the dark sidebar, the highlight color was getting lost against the light gray background

## What are the relevant issue numbers?
Closes #18521 

## Screenshots (if relevant)
![logo](/uploads/3743925862fa8e17d2fec426f7e4c853/logo.gif)

cc @jschatz1 @lbennett @dzaporozhets 

See merge request !4690

Improve pipeline status in case that pipeline has no jobs

## What does this MR do?

This MR resolves problem with pipeline status when there are no build in pipeline.

This can happen when builds were skipped - for example - by using `only`/`except` keyword in `.gitlab-ci.yml`.

## What are the relevant issue numbers?

Closes #17977

See merge request !4403

Turn Group#owners into a has_many association

## What does this MR do?

This turns the regular method `Group#owners` into a `has_many` association.

## Are there points in the code the reviewer needs to double check?

As far as I can tell there's no way to do this without using an intermediate association, but perhaps I'm missing something. The reason an intermediate association is needed is because the supplied Proc is applied to the _final_ association (the one returning users), this means that when using a single `has_many` you can't filter out any intermediate rows (e.g. group members).

## Why was this MR needed?

This code being a regular method would prevent eager loading of the owners of a Group, turning it into a `has_many` association resolves this problem. This was discovered in !4410.

## What are the relevant issue numbers?

None.

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] ~~[Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)~~
- [x] ~~API support added~~
- [ ] Tests
  - [x] Added for this feature/bug
  - [ ] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !4676

Make project_id nullable

See merge request !4701

See merge request !4707

Update CHANGELOG

[ci skip]

See merge request !4695

Revert "Merge branch '18047-event-item-links-dont-look-like-links' into 'master'"

## What does this MR do?

## Are there points in the code the reviewer needs to double check?

## Why was this MR needed?

## What are the relevant issue numbers?

## Screenshots (if relevant)

## Does this MR meet the acceptance criteria?

- [ ] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] API support added
- [ ] Tests
  - [ ] Added for this feature/bug
  - [ ] All builds are passing
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !4692

Pinned sidebar navigation option

## What does this MR do?

Gives the ability to pin the sidebar navigation. Pinning is only possible on screens with above `lg` which is `1200px` if not it reverts to being hidden by default.

## What are the relevant issue numbers?

Closes #18542 

## Screenshots (if relevant)

### Un-pinned

![Screen_Shot_2016-06-15_at_17.29.08](/uploads/82b7495df6ea46a2334a7c25d03b019b/Screen_Shot_2016-06-15_at_17.29.08.png)

### Pinned

![Screen_Shot_2016-06-15_at_17.29.41](/uploads/3eb381be807985d0a9583f000cb802f9/Screen_Shot_2016-06-15_at_17.29.41.png)

See merge request !4683

Improve usability for prioritized labels

## What does this MR do?
- Adds an icon next to prioritized label rows serving as a hint to the user that the labels can be sorted by dragging them.
- Adds description about prioritized labels.
- Restores empty message when removing all prioritized labels.

## Screenshots (if relevant)

![labels](/uploads/1da16c4c54a894fa45ccc47e3a5bdf7a/labels.gif)

Closes #18575 

See merge request !4652

Fix long commit message scroll issue.

Fixes horizontal scrollbar for long commit description.

## Before
![before_](/uploads/d261e26044d8ab451bc7768b16316253/before_.png)

## After
![after](/uploads/f27227a77d25d80b797e3ffc916d1957/after.png)

See merge request !4643