Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
G
gitlab-ce
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
Tatuya Kamada
gitlab-ce
Commits
39a9af15
Commit
39a9af15
authored
Nov 15, 2016
by
Achilleas Pipinellis
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Swap HA setups
parent
cc2bf503
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
45 additions
and
42 deletions
+45
-42
doc/administration/high_availability/README.md
doc/administration/high_availability/README.md
+18
-15
doc/administration/high_availability/redis.md
doc/administration/high_availability/redis.md
+27
-27
No files found.
doc/administration/high_availability/README.md
View file @
39a9af15
...
@@ -7,19 +7,10 @@ highly available.
...
@@ -7,19 +7,10 @@ highly available.
## Architecture
## Architecture
### Active/Passive
There are two kinds of setups:
For pure high-availability/failover with no scaling you can use an
active/passive configuration. This utilizes DRBD (Distributed Replicated
Block Device) to keep all data in sync. DRBD requires a low latency link to
remain in sync. It is not advisable to attempt to run DRBD between data centers
or in different cloud availability zones.
Components/Servers Required:
-
active/active
-
active/passive
-
2 servers/virtual machines (one active/one passive)
![
Active/Passive HA Diagram
](
../img/high_availability/active-passive-diagram.png
)
### Active/Active
### Active/Active
...
@@ -28,12 +19,24 @@ user requests simultaneously. The database, Redis, and GitLab application are
...
@@ -28,12 +19,24 @@ user requests simultaneously. The database, Redis, and GitLab application are
all deployed on separate servers. The configuration is
**only**
highly-available
all deployed on separate servers. The configuration is
**only**
highly-available
if the database, Redis and storage are also configured as such.
if the database, Redis and storage are also configured as such.
![
Active/Active HA Diagram
](
../img/high_availability/active-active-diagram.png
)
Follow the steps below to configure an active/active setup:
**Steps to configure active/active:**
1.
[
Configure the database
](
database.md
)
1.
[
Configure the database
](
database.md
)
1.
[
Configure Redis
](
redis.md
)
1.
[
Configure Redis
](
redis.md
)
1.
[
Configure NFS
](
nfs.md
)
1.
[
Configure NFS
](
nfs.md
)
1.
[
Configure the GitLab application servers
](
gitlab.md
)
1.
[
Configure the GitLab application servers
](
gitlab.md
)
1.
[
Configure the load balancers
](
load_balancer.md
)
1.
[
Configure the load balancers
](
load_balancer.md
)
![
Active/Active HA Diagram
](
../img/high_availability/active-active-diagram.png
)
### Active/Passive
For pure high-availability/failover with no scaling you can use an
active/passive configuration. This utilizes DRBD (Distributed Replicated
Block Device) to keep all data in sync. DRBD requires a low latency link to
remain in sync. It is not advisable to attempt to run DRBD between data centers
or in different cloud availability zones.
Components/Servers Required: 2 servers/virtual machines (one active/one passive)
![
Active/Passive HA Diagram
](
../img/high_availability/active-passive-diagram.png
)
doc/administration/high_availability/redis.md
View file @
39a9af15
...
@@ -16,10 +16,10 @@ Omnibus GitLab packages.
...
@@ -16,10 +16,10 @@ Omnibus GitLab packages.
-
You are highly encouraged to read the
[
Redis Sentinel
][
sentinel
]
documentation
-
You are highly encouraged to read the
[
Redis Sentinel
][
sentinel
]
documentation
before configuring Redis HA with GitLab to fully understand the topology and
before configuring Redis HA with GitLab to fully understand the topology and
architecture.
architecture.
-
This is the documentation for the Omnibus
packages. For installations from
-
This is the documentation for the Omnibus
GitLab packages. For installations
source, follow the
[
Redis HA source install
](
redis_source.md
)
guide.
from
source, follow the
[
Redis HA source install
](
redis_source.md
)
guide.
-
Redis Sentinel
is bundled with Omnibus GitLab Enterprise Edition only. For the
-
Redis Sentinel
daemon is bundled with Omnibus GitLab Enterprise Edition only.
Omnibus Community Edition and installations from source, follow the
For the
Omnibus Community Edition and installations from source, follow the
[
Redis HA source install
](
redis_source.md
)
guide.
[
Redis HA source install
](
redis_source.md
)
guide.
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
...
@@ -50,7 +50,6 @@ Omnibus GitLab packages.
...
@@ -50,7 +50,6 @@ Omnibus GitLab packages.
-
[
Troubleshooting Redis replication
](
#troubleshooting-redis-replication
)
-
[
Troubleshooting Redis replication
](
#troubleshooting-redis-replication
)
-
[
Troubleshooting Sentinel
](
#troubleshooting-sentinel
)
-
[
Troubleshooting Sentinel
](
#troubleshooting-sentinel
)
-
[
Changelog
](
#changelog
)
-
[
Changelog
](
#changelog
)
-
[
Experimental Redis Sentinel support
](
#experimental-redis-sentinel-support
)
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
...
@@ -76,15 +75,16 @@ making sure you have redundant connectivity between Redis / Sentinel and
...
@@ -76,15 +75,16 @@ making sure you have redundant connectivity between Redis / Sentinel and
GitLab instances, otherwise the networks will become a single point of
GitLab instances, otherwise the networks will become a single point of
failure.
failure.
Read carefully how to configure the components below.
Make sure that you read this document once as a whole before configuring the
components below.
### High Availability with Sentinel
### High Availability with Sentinel
>
>
-
S
ince GitLab
`8.11`
, you can configure a list of Redis Sentinel servers that
-
S
tarting with GitLab
`8.11`
, you can configure a list of Redis Sentinel
will monitor a group of Redis servers to provide failover support.
servers that
will monitor a group of Redis servers to provide failover support.
-
With GitLab
`8.14`
, we bundled Redis Sentinel as part of Omnibus package and
-
Starting with GitLab
`8.14`
, the Omnibus GitLab Enterprise Edition package
improved the way you use and configure i
t.
comes with Redis sentinel daemon suppor
t.
High Availability with Redis requires a few things:
High Availability with Redis requires a few things:
...
@@ -136,20 +136,21 @@ the Omnibus GitLab package in `5` **independent** machines, both with
...
@@ -136,20 +136,21 @@ the Omnibus GitLab package in `5` **independent** machines, both with
Based on your infrastructure setup and how you have installed GitLab, there are
Based on your infrastructure setup and how you have installed GitLab, there are
multiple ways to configure Redis HA. Omnibus GitLab packages have Redis and/or
multiple ways to configure Redis HA. Omnibus GitLab packages have Redis and/or
Redis Sentinel bundled with them
to save you the hassle to install them yourself
.
Redis Sentinel bundled with them
so you only need to focus on configuration
.
Pick the one that suits your needs.
Pick the one that suits your needs.
-
[
Installations from source
][
source
]
: You need to install Redis and Sentinel
-
[
Installations from source
][
source
]
: You need to install Redis and Sentinel
yourself. Use the
[
Redis HA source install
](
redis_source.md
)
guide.
yourself. Use the
[
Redis HA installation from source
](
redis_source.md
)
guide.
-
[
Omnibus package Community Edition (CE)
][
ce
]
: Redis is bundled, so you can use the
-
[
Omnibus package Community Edition (CE) package
][
ce
]
: Redis is bundled, so you
package with only the Redis service enabled (works for both master and slave
can use the package with only the Redis service enabled (works for both master
setups).
and slave setups). To install and configure Sentinel, you can use the
-
[
Omnibus package Enterprise Edition (EE)
][
ee
]
: Both Redis and Sentinel are bundled,
[
Redis HA installation from source
](
redis_source.md
)
guide.
so you can use the EE package to setup the whole Redis HA infrastructure
-
[
Omnibus package Enterprise Edition (EE) package
][
ee
]
: Both Redis and Sentinel
(master, slave and Sentinel).
are bundled, so you can use the EE package to setup the whole Redis HA
infrastructure (master, slave and Sentinel).
Note that if you have installed GitLab using the Omnibus packages (both CE and EE),
you can also use an
[
external Redis server
](
#using-a-non-omnibus-external-redis-server
)
.
Note that if you have installed GitLab using the Omnibus GitLab packages (both
CE and EE), you can also use an
[
external Redis server
](
#using-a-non-omnibus-external-redis-server
)
.
### Using a non-Omnibus external Redis server
### Using a non-Omnibus external Redis server
...
@@ -198,7 +199,7 @@ each other over the network.
...
@@ -198,7 +199,7 @@ each other over the network.
Sentinels watch both other sentinels and Redis nodes. Whenever a Sentinel
Sentinels watch both other sentinels and Redis nodes. Whenever a Sentinel
detects that a Redis node is not responding, it will announce that to the
detects that a Redis node is not responding, it will announce that to the
other sentinels.
You
have to reach the
**quorum**
, the minimum amount of
other sentinels.
They
have to reach the
**quorum**
, the minimum amount of
sentinels that agrees that a node is down, to be able to start a failover.
sentinels that agrees that a node is down, to be able to start a failover.
Whenever the
**quorum**
is met, you need the
**majority**
of all known
Whenever the
**quorum**
is met, you need the
**majority**
of all known
...
@@ -267,9 +268,8 @@ This is the section where we install and setup the new Redis instances.
...
@@ -267,9 +268,8 @@ This is the section where we install and setup the new Redis instances.
already have it installed and running, read how to
already have it installed and running, read how to
[
switch from a single-machine installation to Redis HA
](
#switching-from-an-existing-single-machine-installation-to-redis-ha
)
.
[
switch from a single-machine installation to Redis HA
](
#switching-from-an-existing-single-machine-installation-to-redis-ha
)
.
-
Redis nodes (both master and slaves) will need the same password defined in
-
Redis nodes (both master and slaves) will need the same password defined in
`redis['password']`
and
`redis['master_password']`
. At any time during a
`redis['password']`
. At any time during a failover the Sentinels can
failover the Sentinels can reconfigure a node and change its status
reconfigure a node and change its status from master to slave and vice versa.
from master to slave and vice versa.
A summary of what are we going to do:
A summary of what are we going to do:
...
@@ -354,7 +354,7 @@ a failover, as the nodes will be managed by the Sentinels, and even after a
...
@@ -354,7 +354,7 @@ a failover, as the nodes will be managed by the Sentinels, and even after a
`gitlab-ctl reconfigure`
, they will get their configuration restored by
`gitlab-ctl reconfigure`
, they will get their configuration restored by
the same Sentinels.
the same Sentinels.
### Configuring the Sentinel instances
### Configuring the
Redis
Sentinel instances
>**Note:**
>**Note:**
-
Redis Sentinel is bundled with Omnibus GitLab Enterprise Edition only. For the
-
Redis Sentinel is bundled with Omnibus GitLab Enterprise Edition only. For the
...
@@ -425,7 +425,7 @@ redis['enable'] = false
...
@@ -425,7 +425,7 @@ redis['enable'] = false
If you fail to replicate first, you may loose data (unprocessed background jobs).
If you fail to replicate first, you may loose data (unprocessed background jobs).
##
Minimal example
configuration with 1 master, 2 slaves and 3 sentinels
##
Example of a minimal
configuration with 1 master, 2 slaves and 3 sentinels
In this example we consider that all servers have an internal network
In this example we consider that all servers have an internal network
interface with IPs in the
`10.0.0.x`
range, and that they can connect
interface with IPs in the
`10.0.0.x`
range, and that they can connect
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment