Commit 0115f8f0 authored by Achilleas Pipinellis's avatar Achilleas Pipinellis

Merge branch 'sh-geo-use-replication-slots' into 'master'

Update Geo documentation to use PostgreSQL replication slots

Closes #1844

See merge request !2245
parents 8b0ec861 1a7b5e7d
......@@ -31,6 +31,11 @@ connect to `secondary` database servers (which are read-only too).
In many databases documentation you will see `primary` being references as `master`
and `secondary` as either `slave` or `standby` server (read-only).
New for GitLab 9.4: We recommend using [PostgreSQL replication
slots](https://medium.com/@tk512/replication-slots-in-postgresql-b4b03d277c75)
to ensure the primary retains all the data necessary for the secondaries to
recover. See below for more details.
### Prerequisites
The following guide assumes that:
......@@ -73,6 +78,8 @@ The following guide assumes that:
postgresql['listen_address'] = "1.2.3.4"
postgresql['trust_auth_cidr_addresses'] = ['127.0.0.1/32','1.2.3.4/32']
postgresql['md5_auth_cidr_addresses'] = ['5.6.7.8/32']
# New for 9.4: Set this to be the number of Geo secondary nodes you have
postgresql['max_replication_slots'] = 1
# postgresql['max_wal_senders'] = 10
# postgresql['wal_keep_segments'] = 10
```
......@@ -118,6 +125,7 @@ The following guide assumes that:
postgresql['listen_address'] = "10.1.5.3"
postgresql['trust_auth_cidr_addresses'] = ['127.0.0.1/32','10.1.5.3/32']
postgresql['md5_auth_cidr_addresses'] = ['10.1.10.5/32']
postgresql['max_replication_slots'] = 1 # Number of Geo secondary nodes
# postgresql['max_wal_senders'] = 10
# postgresql['wal_keep_segments'] = 10
```
......@@ -138,6 +146,8 @@ The following guide assumes that:
1. Check to make sure your firewall rules are set so that the secondary nodes
can access port 5432 on the primary node.
1. Save the file and [reconfigure GitLab][] for the changes to take effect.
1. New for 9.4: Restart your primary PostgreSQL server to ensure the replication slot changes
take effect (`sudo gitlab-ctl restart postgresql` for Omnibus-provided PostgreSQL).
1. Now that the PostgreSQL server is set up to accept remote connections, run
`netstat -plnt` to make sure that PostgreSQL is listening to the server's
public IP.
......@@ -196,16 +206,25 @@ data before running `pg_basebackup`.
sudo -i
```
1. New for 9.4: Choose a database-friendly name to use for your secondary to use as the
replication slot name. For example, if your domain is
`geo-secondary.mydomain.com`, you may use `geo_secondary_my_domain_com` as
the slot name.
1. Execute the command below to start a backup/restore and begin the replication:
```
gitlab-ctl replicate-geo-database --host=1.2.3.4
gitlab-ctl replicate-geo-database --host=1.2.3.4 --slot-name=geo-secondary_my_domain_com
```
Change the `--host=` to the primary node IP or FQDN. You can check other possible
parameters with `--help`. When prompted, enter the password you set up for
the `gitlab_replicator` user in the first step.
New for 9.4: Change the `--slot-name` to the name of the replication slot
to be used on the primary database. The script will attempt to create the
replication slot automatically if it does not exist.
The replication process is now over.
### Next steps
......
......@@ -68,10 +68,14 @@ The following guide assumes that:
max_wal_senders = 5
min_wal_size = 80MB
max_wal_size = 1GB
max_replicaton_slots = 1 # Number of Geo secondary nodes
wal_keep_segments = 10
hot_standby = on
```
Be sure to set `max_replication_slots` to the number of Geo secondary
nodes that you may potentially have (at least 1).
See the Omnibus notes above for more details of `listen_address`.
You may also want to edit the `wal_keep_segments` and `max_wal_senders` to
......@@ -102,6 +106,22 @@ The following guide assumes that:
```
1. Restart PostgreSQL for the changes to take effect.
1. Choose a database-friendly name to use for your secondary to use as the
replication slot name. For example, if your domain is
`geo-secondary.mydomain.com`, you may use `geo_secondary_my_domain_com` as
the slot name.
1. Create the replication slot on the primary:
```
$ sudo -u postgres psql -c "SELECT * FROM pg_create_physical_replication_slot('geo_secondary_my_domain');"
slot_name | xlog_position
-------------------------+---------------
geo_secondary_my_domain |
(1 row)
```
1. Now that the PostgreSQL server is set up to accept remote connections, run
`netstat -plnt` to make sure that PostgreSQL is listening to the server's
public IP.
......
......@@ -50,6 +50,21 @@ where you have to fix (all commands and path locations are for Omnibus installs)
# remove old entries to your primary gitlab in known_hosts
ssh-keyscan -R your-primary-gitlab.example.com
- How do I fix the message, "ERROR: replication slots can only be used if max_replication_slots > 0"?
- This means that the `max_replication_slots` PostgreSQL variable needs to
be set on the primary database. In GitLab 9.4, we have made this setting
default to 1. You may need to increase this value if you have more Geo
secondary nodes. Be sure to restart PostgreSQL for this to take
effect. See the [PostgreSQL replication
setup](database.md#postgresql-replication) guide for more details.
- How do I fix the message, "FATAL: could not start WAL streaming: ERROR: replication slot "geo_secondary_my_domain_com" does not exist"?
- This occurs when PostgreSQL does not have a replication slot for the
secondary by that name. You may want to rerun the [replication
process](database.md) on the secondary.
Visit the primary node's **Admin Area ➔ Geo Nodes** (`/admin/geo_nodes`) in
your browser. We perform the following health checks on each secondary node
to help identify if something is wrong:
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment