Skip to content

G
gitlab-ce
Commit ffd58246 authored by Ian Baum's avatar Ian Baum
Browse files
Options

Update PostgreSQL HA alpha document for repmgr automation 

This reflects the current state of the repmgr in omnibus-gitlab MR 1664
parent 8e03b701
Hide whitespace changes
Inline Side-by-side
Showing with 17 additions and 108 deletions
doc/administration/high_availability/alpha_database.md
View file @ ffd58246
...@@ -57,56 +57,18 @@ using [repmgr](http://www.repmgr.org/) to handle standby synchronization, and fa ...@@ -57,56 +57,18 @@ using [repmgr](http://www.repmgr.org/) to handle standby synchronization, and fa
mailroom['enable'] = false mailroom['enable'] = false
# PostgreSQL configuration # PostgreSQL configuration
postgresql['md5_auth_cidr_addresses'] = ['0.0.0.0/0'] postgresql['md5_auth_cidr_addresses'] = %w(0.0.0.0/0)
postgresql['listen_address'] = '0.0.0.0' postgresql['listen_address'] = '0.0.0.0'
postgresql['sql_user_password'] = 'PASSWORD_HASH' # This is the hash generated in the previous step postgresql['sql_user_password'] = 'PASSWORD_HASH' # This is the hash generated in the previous step
postgresql['trust_auth_cidr_addresses'] = ['127.0.0.0/24'] postgresql['trust_auth_cidr_addresses'] = %w(127.0.0.0/24)
postgresql['hot_standby'] = 'on' postgresql['hot_standby'] = 'on'
postgresql['wal_level'] = 'replica' postgresql['wal_level'] = 'replica'
postgresql['max_wal_senders'] = X # Should be set to at least 1 more than the number of nodes in the cluster postgresql['max_wal_senders'] = X # Should be set to at least 1 more than the number of nodes in the cluster
postgresql['shared_preload_libraries'] = 'repmgr_funcs' # If this attribute is already defined, append the new value as a comma separated list postgresql['shared_preload_libraries'] = 'repmgr_funcs' # If this attribute is already defined, append the new value as a comma separated list
postgresql['custom_pg_hba_entries']['repmgr'] = [
{ # repmgr configuration
type: 'local', repmgr['enable'] = true
database: 'replication', repmgr['trust_auth_cidr_addresses'] = %w(XXX.XXX.XXX.XXX/YY) # This should be the CIDR of the network your database nodes are on
user: 'gitlab_replicator',
method: 'trust',
},
{
type: 'host',
database: 'replication',
user: 'gitlab_replicator',
cidr: '127.0.0.1/32',
method: 'trust'
},
{
type: 'host',
database: 'replication',
user: 'gitlab_replicator',
cidr: 'XXX.XXX.XXX.XXX/YY', # This should be the CIDR of the network your database nodes are on
method: 'trust'
},
{
type: 'local',
database: 'repmgr',
user: 'gitlab_replicator',
method: 'trust',
},
{
type: 'host',
database: 'repmgr',
user: 'gitlab_replicator',
cidr: '127.0.0.1/32',
method: 'trust'
},
{
type: 'host',
database: 'repmgr',
user: 'gitlab_replicator',
cidr: 'XXX.XXX.XXX.XXX/YY', # This should be the CIDR of the network your database nodes are on
method: 'trust'
}
]
# Disable automatic database migrations # Disable automatic database migrations
gitlab_rails['auto_migrate'] = false gitlab_rails['auto_migrate'] = false
...@@ -117,20 +79,6 @@ using [repmgr](http://www.repmgr.org/) to handle standby synchronization, and fa ...@@ -117,20 +79,6 @@ using [repmgr](http://www.repmgr.org/) to handle standby synchronization, and fa
# gitlab-ctl reconfigure # gitlab-ctl reconfigure
``` ```
1. Create `/var/opt/gitlab/postgresql/repmgr.conf` with the following content. Use a unique integer for the value of node.
```
cluster=gitlab_cluster
node=X
node_name=HOSTNAME
conninfo='host=HOSTNAME user=gitlab_replicator dbname=repmgr'
pg_bindir='/opt/gitlab/embedded/bin'
service_start_command = '/opt/gitlab/bin/gitlab-ctl start postgresql'
service_stop_command = '/opt/gitlab/bin/gitlab-ctl stop postgresql'
service_restart_command = '/opt/gitlab/bin/gitlab-ctl restart postgresql'
promote_command = '/opt/gitlab/embedded/bin/repmgr standby promote -f /var/opt/gitlab/postgresql/repmgr.conf'
follow_command = '/opt/gitlab/embedded/bin/repmgr standby follow -f /var/opt/gitlab/postgresql/repmgr.conf'
```
### On the primary database node ### On the primary database node
1. Open a database prompt: 1. Open a database prompt:
...@@ -157,12 +105,6 @@ using [repmgr](http://www.repmgr.org/) to handle standby synchronization, and fa ...@@ -157,12 +105,6 @@ using [repmgr](http://www.repmgr.org/) to handle standby synchronization, and fa
Enter it again: Enter it again:
``` ```
1. Create the repmgr database:
```
template1=# ALTER USER gitlab_replicator WITH SUPERUSER;
template1=# CREATE DATABASE repmgr WITH OWNER gitlab_replicator;
```
1. Switch to the GitLab database and Enable the `pg_trgm` extension: 1. Switch to the GitLab database and Enable the `pg_trgm` extension:
``` ```
template1=# \c gitlabhq_production template1=# \c gitlabhq_production
...@@ -175,56 +117,27 @@ using [repmgr](http://www.repmgr.org/) to handle standby synchronization, and fa ...@@ -175,56 +117,27 @@ using [repmgr](http://www.repmgr.org/) to handle standby synchronization, and fa
1. Exit the database prompt by typing `\q` and Enter. 1. Exit the database prompt by typing `\q` and Enter.
1. Register the node as the initial master node for the repmgr cluster
```
# su - gitlab-psql
$ repmgr -f /var/opt/gitlab/postgresql/repmgr.conf master register
NOTICE: master node correctly registered for cluster 'gitlab_cluster' with id X (conninfo: host=HOSTNAME user=gitlab_replicator dbname=repmgr)
```
1. Verify the cluster is initialized with one node 1. Verify the cluster is initialized with one node
``` ```
$ repmgr -f /var/opt/gitlab/postgresql/repmgr.conf cluster show # gitlab-ctl repmgr cluster show
Role | Name | Upstream | Connection String Role | Name | Upstream | Connection String
----------+-------------|----------|---------------------------------------- ----------+-------------|----------|----------------------------------------
* master | HOSTNAME | | host=HOSTNAME user=gitlab_replicator dbname=repmgr * master | HOSTNAME | | host=HOSTNAME user=gitlab_repmgr dbname=gitlab_repmgr
``` ```
### On each standby node ### On each standby node
1. Stop postgresql 1. Setup the repmgr standby
```
# gitlab-ctl stop postgresql
```
1. Clear out the current data directory
```
# rm -rf /var/opt/gitlab/postgresql/data/*
```
1. Synchronize the data from the primary node:
```
# su - gitlab-psql
$ repmgr -h PRIMARY_HOSTNAME -U gitlab_replicator -d repmgr -D /var/opt/gitlab/postgresql/data/ -f /var/opt/gitlab/postgresql/repmgr.conf standby clone
```
1. Start the database
```
$ gitlab-ctl start postgresql
```
1. Register the node with the cluster
``` ```
$ repmgr -f /var/opt/gitlab/postgresql/repmgr.conf standby register # gitlab-ctl repmgr standby setup MASTER_NODE
NOTICE: standby node correctly registered for cluster gitlab_cluster with id X (conninfo: host=HOSTNAME user=gitlab_replicator dbname=repmgr)
``` ```
1. Verify the node now appears in the cluster 1. Verify the node now appears in the cluster
``` ```
$ repmgr -f /var/opt/gitlab/postgresql/repmgr.conf cluster show # gitlab-ctl repmgr cluster show
Role | Name | Upstream | Connection String Role | Name | Upstream | Connection String
----------+------------|------------|------------------------------------------------ ----------+------------|------------|------------------------------------------------
* master | MASTER | | host=MASTER_HOSTNAME user=gitlab_replicator dbname=repmgr * master | MASTER | | host=MASTER_HOSTNAME user=gitlab_repmgr dbname=gitlab_repmgr
standby | STANDBY | MASTER | host=STANDBY_HOSTNAME user=gitlab_replicator dbname=repmgr standby | STANDBY | MASTER | host=STANDBY_HOSTNAME user=gitlab_repmgr dbname=gitlab_repmgr
``` ```
### (Optional) Enable repmgrd ### (Optional) Enable repmgrd
...@@ -257,14 +170,12 @@ If your master node is experiencing an issue, you can manually failover. ...@@ -257,14 +170,12 @@ If your master node is experiencing an issue, you can manually failover.
1. Login to the server that should become the new master and run the following 1. Login to the server that should become the new master and run the following
``` ```
# su - gitlab-psql # gitlab-ctl repmgr standby promote
$ repmgr -f /var/opt/gitlab/postgresql/repmgr.conf standby promote
``` ```
1. If there are any other standby servers in the cluster, have them follow the new master server 1. If there are any other standby servers in the cluster, have them follow the new master server
``` ```
# su - gitlab-psql # gitlab-ctl repmgr standby follow NEW_MASTER
# repmgr -f /var/opt/gitlab/postgresql/repmgr.conf -h NEW_MASTER -U gitlab_replicator -d repmgr -d /var/opt/gitlab/postgresql/data standby follow
``` ```
1. On the servers that run `gitlab-rails`, set the `gitlab_rails['db_host']` attribute to the new master, and run `gitlab-ctl reconfigure` 1. On the servers that run `gitlab-rails`, set the `gitlab_rails['db_host']` attribute to the new master, and run `gitlab-ctl reconfigure`
...@@ -273,14 +184,12 @@ If your master node is experiencing an issue, you can manually failover. ...@@ -273,14 +184,12 @@ If your master node is experiencing an issue, you can manually failover.
1. If you want to remove the node from the cluster, on any other node in the cluster, run: 1. If you want to remove the node from the cluster, on any other node in the cluster, run:
``` ```
# su - gitlab-psql # gitlab-ctl repmgr standby unregister --node=X # X should be the value of node in repmgr.conf on the old server
$ repmgr -f /var/opt/gitlab/postgresql/repmgr.conf standby unregister --node=X # X should be the value of node in repmgr.conf on the old server
``` ```
1. If the failed master has been recovered, it can be converted to a standby server and follow the new master server[^1] 1. If the failed master has been recovered, it can be converted to a standby server and follow the new master server[^1]
``` ```
# su - gitlab-psql # gitlab-ctl repmgr standby follow NEW_MASTER
# repmgr -f /var/opt/gitlab/postgresql/repmgr.conf -h NEW_MASTER -U gitlab_replicator -d repmgr -d /var/opt/gitlab/postgresql/data standby follow
``` ```
[^1]: When the server is back online, and before you switch it to a standby node, repmgr will report that there are two masters. [^1]: When the server is back online, and before you switch it to a standby node, repmgr will report that there are two masters.
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7