Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
G
gitlab-ce
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
1
Merge Requests
1
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
nexedi
gitlab-ce
Commits
7a19879d
Commit
7a19879d
authored
Apr 29, 2019
by
Hossein Pursultani
Committed by
Robert Speicher
Apr 29, 2019
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Move sidekiq-cluster to ee/bin
parent
c5c6e639
Changes
5
Hide whitespace changes
Inline
Side-by-side
Showing
5 changed files
with
20 additions
and
14 deletions
+20
-14
doc/administration/operations/extra_sidekiq_processes.md
doc/administration/operations/extra_sidekiq_processes.md
+7
-7
ee/bin/sidekiq-cluster
ee/bin/sidekiq-cluster
+4
-3
ee/changelogs/unreleased/6565-move-sidekiq-cluster.yml
ee/changelogs/unreleased/6565-move-sidekiq-cluster.yml
+5
-0
ee/spec/bin/sidekiq_cluster_spec.rb
ee/spec/bin/sidekiq_cluster_spec.rb
+2
-2
lib/gitlab/sidekiq_config.rb
lib/gitlab/sidekiq_config.rb
+2
-2
No files found.
doc/administration/operations/extra_sidekiq_processes.md
View file @
7a19879d
...
@@ -93,11 +93,11 @@ This tells the additional processes how often to check for enqueued jobs.
...
@@ -93,11 +93,11 @@ This tells the additional processes how often to check for enqueued jobs.
## Starting extra processes via command line
## Starting extra processes via command line
Starting extra Sidekiq processes can be done using the command
Starting extra Sidekiq processes can be done using the command
`/opt/gitlab/embedded/service/gitlab-rails/bin/sidekiq-cluster`
. This command
`/opt/gitlab/embedded/service/gitlab-rails/
ee/
bin/sidekiq-cluster`
. This command
takes arguments using the following syntax:
takes arguments using the following syntax:
```
bash
```
bash
/opt/gitlab/embedded/service/gitlab-rails/bin/sidekiq-cluster
[
QUEUE,QUEUE,...]
[
QUEUE, ...]
/opt/gitlab/embedded/service/gitlab-rails/
ee/
bin/sidekiq-cluster
[
QUEUE,QUEUE,...]
[
QUEUE, ...]
```
```
Each separate argument denotes a group of queues that have to be processed by a
Each separate argument denotes a group of queues that have to be processed by a
...
@@ -115,14 +115,14 @@ For example, say you want to start 2 extra processes: one to process the
...
@@ -115,14 +115,14 @@ For example, say you want to start 2 extra processes: one to process the
done as follows:
done as follows:
```
bash
```
bash
/opt/gitlab/embedded/service/gitlab-rails/bin/sidekiq-cluster process_commit post_receive
/opt/gitlab/embedded/service/gitlab-rails/
ee/
bin/sidekiq-cluster process_commit post_receive
```
```
If you instead want to start one process processing both queues you'd use the
If you instead want to start one process processing both queues you'd use the
following syntax:
following syntax:
```
bash
```
bash
/opt/gitlab/embedded/service/gitlab-rails/bin/sidekiq-cluster process_commit,post_receive
/opt/gitlab/embedded/service/gitlab-rails/
ee/
bin/sidekiq-cluster process_commit,post_receive
```
```
If you want to have one Sidekiq process process the "process_commit" and
If you want to have one Sidekiq process process the "process_commit" and
...
@@ -130,7 +130,7 @@ If you want to have one Sidekiq process process the "process_commit" and
...
@@ -130,7 +130,7 @@ If you want to have one Sidekiq process process the "process_commit" and
you'd use the following:
you'd use the following:
```
bash
```
bash
/opt/gitlab/embedded/service/gitlab-rails/bin/sidekiq-cluster process_commit,post_receive gitlab_shell
/opt/gitlab/embedded/service/gitlab-rails/
ee/
bin/sidekiq-cluster process_commit,post_receive gitlab_shell
```
```
### Monitoring
### Monitoring
...
@@ -162,7 +162,7 @@ file is written, but this can be changed by passing the `--pidfile` option to
...
@@ -162,7 +162,7 @@ file is written, but this can be changed by passing the `--pidfile` option to
`sidekiq-cluster`
. For example:
`sidekiq-cluster`
. For example:
```
bash
```
bash
/opt/gitlab/embedded/service/gitlab-rails/bin/sidekiq-cluster
--pidfile
/var/run/gitlab/sidekiq_cluster.pid process_commit
/opt/gitlab/embedded/service/gitlab-rails/
ee/
bin/sidekiq-cluster
--pidfile
/var/run/gitlab/sidekiq_cluster.pid process_commit
```
```
Keep in mind that the PID file will contain the PID of the
`sidekiq-cluster`
Keep in mind that the PID file will contain the PID of the
`sidekiq-cluster`
...
@@ -199,7 +199,7 @@ one thread per queue up to a maximum of 50. If you wish to change the cap, use
...
@@ -199,7 +199,7 @@ one thread per queue up to a maximum of 50. If you wish to change the cap, use
the
`-m N`
option. For example, this would cap the maximum number of threads to 1:
the
`-m N`
option. For example, this would cap the maximum number of threads to 1:
```
bash
```
bash
/opt/gitlab/embedded/service/gitlab-rails/bin/sidekiq-cluster process_commit,post_receive
-m
1
/opt/gitlab/embedded/service/gitlab-rails/
ee/
bin/sidekiq-cluster process_commit,post_receive
-m
1
```
```
For each queue group, the concurrency factor will be set to min(number of
For each queue group, the concurrency factor will be set to min(number of
...
...
bin/sidekiq-cluster
→
ee/
bin/sidekiq-cluster
View file @
7a19879d
#!/usr/bin/env ruby
#!/usr/bin/env ruby
# frozen_string_literal: true
require
'optparse'
require
'optparse'
require_relative
'../lib/gitlab/sidekiq_config'
require_relative
'../
../
lib/gitlab/sidekiq_config'
require_relative
'../
ee/
lib/gitlab/sidekiq_cluster'
require_relative
'../lib/gitlab/sidekiq_cluster'
require_relative
'../
ee/
lib/gitlab/sidekiq_cluster/cli'
require_relative
'../lib/gitlab/sidekiq_cluster/cli'
Thread
.
abort_on_exception
=
true
Thread
.
abort_on_exception
=
true
...
...
ee/changelogs/unreleased/6565-move-sidekiq-cluster.yml
0 → 100644
View file @
7a19879d
---
title
:
Move sidekiq-cluster to ee/bin
merge_request
:
11001
author
:
type
:
other
ee/spec/bin/sidekiq_cluster_spec.rb
View file @
7a19879d
require
'spec_helper'
require
'spec_helper'
describe
'bin/sidekiq-cluster'
do
describe
'
ee/
bin/sidekiq-cluster'
do
it
'runs successfully'
,
:aggregate_failures
do
it
'runs successfully'
,
:aggregate_failures
do
cmd
=
%w[bin/sidekiq-cluster --dryrun --negate cronjob]
cmd
=
%w[
ee/
bin/sidekiq-cluster --dryrun --negate cronjob]
output
,
status
=
Gitlab
::
Popen
.
popen
(
cmd
,
Rails
.
root
.
to_s
)
output
,
status
=
Gitlab
::
Popen
.
popen
(
cmd
,
Rails
.
root
.
to_s
)
...
...
lib/gitlab/sidekiq_config.rb
View file @
7a19879d
...
@@ -7,7 +7,7 @@ module Gitlab
...
@@ -7,7 +7,7 @@ module Gitlab
module
SidekiqConfig
module
SidekiqConfig
QUEUE_CONFIG_PATHS
=
%w[app/workers/all_queues.yml ee/app/workers/all_queues.yml]
.
freeze
QUEUE_CONFIG_PATHS
=
%w[app/workers/all_queues.yml ee/app/workers/all_queues.yml]
.
freeze
# This method is called by `bin/sidekiq-cluster` in EE, which runs outside
# This method is called by `
ee/
bin/sidekiq-cluster` in EE, which runs outside
# of bundler/Rails context, so we cannot use any gem or Rails methods.
# of bundler/Rails context, so we cannot use any gem or Rails methods.
def
self
.
worker_queues
(
rails_path
=
Rails
.
root
.
to_s
)
def
self
.
worker_queues
(
rails_path
=
Rails
.
root
.
to_s
)
@worker_queues
||=
{}
@worker_queues
||=
{}
...
@@ -19,7 +19,7 @@ module Gitlab
...
@@ -19,7 +19,7 @@ module Gitlab
end
end
end
end
# This method is called by `bin/sidekiq-cluster` in EE, which runs outside
# This method is called by `
ee/
bin/sidekiq-cluster` in EE, which runs outside
# of bundler/Rails context, so we cannot use any gem or Rails methods.
# of bundler/Rails context, so we cannot use any gem or Rails methods.
def
self
.
expand_queues
(
queues
,
all_queues
=
self
.
worker_queues
)
def
self
.
expand_queues
(
queues
,
all_queues
=
self
.
worker_queues
)
return
[]
if
queues
.
empty?
return
[]
if
queues
.
empty?
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment