Commit b158fe73 authored by Valery Sizov's avatar Valery Sizov Committed by Nick Thomas

Resolve "Document manual disaster recovery process for systems with multiple secondaries"

parent 50d8578f
......@@ -174,13 +174,75 @@ To bring a new secondary online, follow the [Geo setup instructions][setup-geo].
## Promoting secondary Geo replica in multi-secondary configurations
CAUTION: **Warning:**
Disaster Recovery for multi-secondary configurations is in **Alpha** development.
Do not use this as your only Disaster Recovery strategy as you may lose data.
If you have more than one secondary and you need to promote one of them we suggest you to follow
[Promoting secondary Geo replica in single-secondary configurations](#promoting-secondary-geo-replica-in-single-secondary-configurations)
and after that you also need two extra steps.
### Step 1. Prepare the new primary to serve one or more secondaries
1. SSH into your **secondary** and login as root:
```bash
sudo -i
```
1. Edit `/etc/gitlab/gitlab.rb`
```ruby
##
# Primary and Secondary addresses
# - replace '1.2.3.4' with the primary public or VPC address
# - replace '5.6.7.8' with the secondary public or VPC address
##
postgresql['md5_auth_cidr_addresses'] = ['1.2.3.4/32', '5.6.7.8/32']
# Every secondary server needs to have its own slot so specify the number of secondary nodes you're going to have
postgresql['max_replication_slots'] = 1
##
## Disable automatic database migrations temporarily
## (until PostgreSQL is restarted and listening on the private address).
##
gitlab_rails['auto_migrate'] = false
```
For more details about these settings you can read [Configure the primary server][configure-the-primary-server]
1. Save the file and reconfigure GitLab for the database listen changes and
the replication slot changes to be applied.
```bash
gitlab-ctl reconfigure
```
Restart PostgreSQL for its changes to take effect:
```bash
gitlab-ctl restart postgresql
```
1. Re-enable migrations now that PostgreSQL is restarted and listening on the
private address.
Edit `/etc/gitlab/gitlab.rb` and **change** the configuration to `true`:
```ruby
gitlab_rails['auto_migrate'] = true
```
Save the file and reconfigure GitLab:
```bash
gitlab-ctl reconfigure
```
### Step 2. Initiate the replication process
Now we need to make each secondary listen to changes on the new primary. To do that you need
to [initiate the replication process][initiate-the-replication-process] again but this time
for another primary. All the old replication settings will be overwritten.
Disaster Recovery does not yet support systems with multiple
secondary Geo replicas (e.g., one primary and two or more secondaries). We are
working on it, see [gitlab-org/gitlab-ee#4284] for details.
## Troubleshooting
......@@ -206,3 +268,5 @@ section to resolve the error. Otherwise, the secret is lost and you'll need to
[sec-tfa]: ../../../security/two_factor_authentication.md#disabling-2fa-for-everyone
[gitlab-org/omnibus-gitlab#3058]: https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3058
[gitlab-org/gitlab-ee#4284]: https://gitlab.com/gitlab-org/gitlab-ee/issues/4284
[initiate-the-replication-process]: ../replication/database.html#step-3-initiate-the-replication-process
[configure-the-primary-server]: ../replication/database.html#step-1-configure-the-primary-server
---
title: Document manual disaster recovery process for systems with multiple secondaries
merge_request:
author:
type: changed
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment