CE upstream: Pre-9.4 RC1

Closes #2890, #2878, #2716, gitlab-ce#34847, gitaly#355, #2839, and gitlab-ce#32568

See merge request !2390

Closes #2890

[EE] Support multiple Redis instances based on queue type

See merge request !2312

Fix bad arguments passed to BackgroundMigrationWorker

See merge request !2398

According to the docs we should be passing an Array of Arrays:
https://gitlab.com/gitlab-org/gitlab-ee/blob/661a324d578be01742ec2d77bf0adeb59c69faea/app/workers/background_migration_worker.rb#L5-10

Implement progressive elasticsearch indexing for project mirrors

Closes #2881

See merge request !2393

* 9-4-stable: (41 commits)
  Also inject new route helpers into includers of GitlabRoutingHelper
  Remove many N+1 queries with merge requests API
  Change `sign_out` usage to `gitlab_sign_out` in one spec
  Simplify stage_id migration as we now use relations
  Do not schedule bg migration when it is not needed
  Schedule stage_id bg migrations in batches of 10
  Use new `each_batch` helper instead of custom one
  Revert recent changes in migration helpers
  Add some comments on new migrations helpers
  Schedule stage_id background migration in ranges
  Extract `execute_in_batches` migrations helper
  Add walk_table_in_batches and refactor migration helpers
  Reduce a delay between stage_id scheduled migrations
  Improve exception description in bg migrations
  Do not override original AR5 batching interface
  Sanitize id value passed to async background migration
  Improve code examples in background migrations docs
  Add description to exception in bg migrations worker
  Do not compare float with integer in migrations specs
  Improve readability of build stage id migration query
  ...

Conflicts:
	spec/features/boards/boards_spec.rb
	spec/features/issues/create_issue_for_single_discussion_in_merge_request_spec.rb
	spec/features/issues/user_uses_slash_commands_spec.rb
	spec/features/merge_requests/edit_mr_spec.rb
	spec/features/milestones/show_spec.rb
	spec/features/protected_tags_spec.rb

Port "Add variables to pipelines schedules" to EE

See merge request !2383

Merge remote-tracking branch 'origin-ee/feature/intermediate/32568-adding-variables-to-pipelines-schedules-ee' into ce-to-ee-2017-07-08

Port "Add Group secret variables" to EE

See merge request !2373

Show Slack button when there is no team added

See merge request !2381

Fix GitLab Slack application configuration

See merge request !2392

Exclude gitlab_git_test from docs only changes

See merge request !2371

Mark a subgroup-related spec as PostgreSQL-only

Closes #34847

See merge request !12722

Basic BE change

Fix static-snalysis

Move the precedence of group secure variable before project secure variable. Allow project_id to be null.

Separate Ci::VariableProject and Ci::VariableGroup

Add the forgotton files

Add migration file to update type of ci_variables

Fix form_for fpr VariableProject

Fix test

Change the table structure according to the yorik advice

Add necessary migration files. Remove unnecessary migration spec.

Revert safe_model_attributes.yml

Fix models

Fix spec

Avoid self.variable. Use becomes for correct routing.

Use unique index on group_id and key

Add null: false for t.timestamps

Fix schema version

Rename VariableProject and VariableGroup to ProjectVariable and GroupVariable

Rename the rest of them

Add the rest of files

Basic BE change

Fix static-snalysis

Move the precedence of group secure variable before project secure variable. Allow project_id to be null.

Separate Ci::VariableProject and Ci::VariableGroup

Add the forgotton files

Add migration file to update type of ci_variables

Fix form_for fpr VariableProject

Fix test

Change the table structure according to the yorik advice

Add necessary migration files. Remove unnecessary migration spec.

Revert safe_model_attributes.yml

Fix models

Fix spec

Avoid self.variable. Use becomes for correct routing.

Use unique index on group_id and key

Add null: false for t.timestamps

Fix schema version

Rename VariableProject and VariableGroup to ProjectVariable and GroupVariable

Rename the rest of them

Add the rest of files

Implement CURD

Rename codes related to VariableGroup and VariableProject FE part

Remove unneccesary changes

Make Fe code up-to-date

Add protected flag to migration file

Protected group variables essential package

Update schema

Improve doc

Fix logic and spec for models

Fix logic and spec for controllers

 Fix logic and spec for views(pre feature)

Add feature spec

Fixed bugs. placeholder. reveal button. doc.

Add changelog

Remove unnecessary comment

godfat nice catches

Improve secret_variables_for arctecture

Fix spec

Fix StaticAnlysys & path_regex spec

Revert "Improve secret_variables_for arctecture"

This reverts commit c3216ca212322ecf6ca534cb12ce75811a4e77f1.

Use ayufan suggestion for secret_variables_for

Use find instead of find_by

Fix spec message for variable is invalid

Fix spec remove variable.group_id = group.id

godffat spec nitpicks

Use include Gitlab::Routing.url_helpers for presenter spec

Copyedit docs for group-level secret variables

[ci skip]

Wrap additional routes by dash(-). And remove those routes from path_regex.rb.

Add dash for GROUP_ROUTES

Fix variables_controller.rb and format

ayufan nice catches

Use ancestors for avoiding N queries

Add CASE When Clause for saving order when using where IN

secret_variables_for: rails readability versino

Adopt project_variable_path instead of namespace_project_variable_path. (Resolve confilct from master)

gb nice catches

Use new project_variables_path in this MR

Fix to Variable was successfully created

Use variable_params && variable_params_attributes in project variables_controller.rb

Name as variable_params like project variable controller

[EE] Change gitlab_sign_in to sign_in

See merge request !2255

Change gitlab_sign_in to sign_in

See merge request !12369

Also inject new route helpers into includers of GitlabRoutingHelper

Closes #34859

See merge request !12735

Add build stage_id reference background migration

Closes #34151

See merge request !12513