Skip to content

G
gitlab-ce
Commit b3288eec authored by Achilleas Pipinellis's avatar Achilleas Pipinellis
Browse files
Options

Merge branch '1664-geo-improve-setup' into 'master' 

Geo improved setup for 9.1

See merge request !1584
parents 5003d0b7 5b2bceec
Hide whitespace changes
Inline Side-by-side
Showing with 32 additions and 138 deletions
doc/gitlab-geo/configuration.md
View file @ b3288eec
......@@ -62,25 +62,15 @@ logins opened on all nodes as we will be moving back and forth.
sudo -i
```
1. Get the contents of `id_rsa.pub` for the git user:
1. Execute the command below to define the node as primary Geo node:
```
# Omnibus GitLab installations
sudo -u git cat /var/opt/gitlab/.ssh/id_rsa.pub
gitlab-ctl set-geo-primary-node
```
Read more in [additional info for SSH key pairs](#additional-information-for-the-ssh-key-pairs).
1. Visit the primary node's **Admin Area ➔ Geo Nodes** (`/admin/geo_nodes`) in
your browser.
This command will use your defined `external_url` in `gitlab.rb` and pre-generated SSH key pairs.
1. Add the primary node by providing its full URL and the public SSH key
you created previously. Make sure to check the box 'This is a primary node'
when adding it.
![Add new primary Geo node](img/geo_nodes_add_new.png)
1. Click the **Add node** button.
Read more in [additional info for SSH key pairs](#additional-information-for-the-ssh-key-pairs).
### Step 2. Updating the `known_hosts` file of the secondary nodes
......@@ -103,7 +93,6 @@ logins opened on all nodes as we will be moving back and forth.
1. Verify that the fingerprint was added by checking `known_hosts`:
```
# Omnibus GitLab installations
cat /var/opt/gitlab/.ssh/known_hosts
```
......@@ -119,11 +108,10 @@ sensitive data in the database. Any secondary node must have the
sudo -i
```
1. Find the value of `db_key_base` and copy it:
1. Execute the command below to display current encryption key and copy it:
```
# Omnibus GitLab installations
cat /etc/gitlab/gitlab-secrets.json | grep db_key_base
gitlab-rake geo:db:show_encryption_key
```
1. SSH into the **secondary** node and login as root:
......@@ -136,7 +124,6 @@ sensitive data in the database. Any secondary node must have the
previous step:
```
# Omnibus GitLab installations
editor /etc/gitlab/gitlab-secrets.json
```
......@@ -150,69 +137,22 @@ sensitive data in the database. Any secondary node must have the
sudo -i
```
1. (This step is required only if you want to enable the new Disaster Recovery
feature in Alpha shipped in GitLab 9.0) Edit `/etc/gitlab/gitlab.rb`:
1. Edit `/etc/gitlab/gitlab.rb`:
```
geo_postgresql['enable'] = true
```
1. (This step is required only if you want to enable the new Disaster Recovery
feature in Alpha shipped in GitLab 9.0) Create `database_geo.yml` with the
information of your secondary PostgreSQL database. Note that GitLab will
set up another database instance separate from the primary, since this is
where the secondary will track its internal state:
```
sudo cp /opt/gitlab/embedded/service/gitlab-rails/config/database_geo.yml.postgresql /opt/gitlab/embedded/service/gitlab-rails/config/database_geo.yml
```
1. (This step is required only if you want to enable the new Disaster Recovery
feature in Alpha shipped in GitLab 9.0) Edit the content of
`database_geo.yml` in `production:` to be like the following:
```yaml
#
# PRODUCTION
#
production:
adapter: postgresql
encoding: unicode
database: gitlabhq_geo_production
pool: 10
username: gitlab_geo
# password:
host: /var/opt/gitlab/geo-postgresql
port: 5431
```
1. (This step is required only if you want to enable the new Disaster Recovery
feature in Alpha shipped in GitLab 9.0) Reconfigure GitLab:
1. Reconfigure GitLab:
```
sudo gitlab-ctl reconfigure
```
1. (This step is required only if you want to enable the new Disaster Recovery
feature in Alpha shipped in GitLab 9.0) Set up the Geo tracking database:
```
sudo gitlab-rake geo:db:setup
```
1. Create a new SSH key pair for the secondary node. Choose the default location
and leave the password blank by hitting 'Enter' three times:
```bash
sudo -u git -H ssh-keygen -b 4096 -C 'Secondary GitLab Geo node'
```
Read more in [additional info for SSH key pairs](#additional-information-for-the-ssh-key-pairs).
1. Get the contents of `id_rsa.pub` the was just created:
1. Get the contents of `id_rsa.pub` key that was pre-generated by Omnibus GitLab
and copy them:
```
# Omnibus installations
sudo -u git cat /var/opt/gitlab/.ssh/id_rsa.pub
```
......
doc/gitlab-geo/configuration_source.md
View file @ b3288eec
......@@ -75,7 +75,6 @@ logins opened on all nodes as we will be moving back and forth.
1. Get the contents of `id_rsa.pub` for the git user:
```
# Installations from source
sudo -u git cat /home/git/.ssh/id_rsa.pub
```
......@@ -111,7 +110,6 @@ logins opened on all nodes as we will be moving back and forth.
1. Verify that the fingerprint was added by checking `known_hosts`:
```
# Installations from source
cat /home/git/.ssh/known_hosts
```
......@@ -127,11 +125,10 @@ sensitive data in the database. Any secondary node must have the
sudo -i
```
1. Find the value of `db_key_base` and copy it:
1. Execute the command below to display the current encryption key and copy it:
```
# Installations from source
cat /home/git/gitlab/config/secrets.yml | grep db_key_base
bundle exec rake geo:db:show_encryption_key
```
1. SSH into the **secondary** node and login as root:
......@@ -184,7 +181,6 @@ sensitive data in the database. Any secondary node must have the
previous step:
```
# Installations from source
editor /home/git/gitlab/config/secrets.yml
```
......@@ -210,7 +206,6 @@ sensitive data in the database. Any secondary node must have the
1. Get the contents of `id_rsa.pub` the was just created:
```
# Installations from source
sudo -u git cat /home/git/.ssh/id_rsa.pub
```
......
doc/gitlab-geo/database.md
View file @ b3288eec
......@@ -66,14 +66,12 @@ The following guide assumes that:
1. Edit `/etc/gitlab/gitlab.rb` and add the following:
```ruby
geo_primary_role['enable'] = true
postgresql['listen_address'] = "1.2.3.4"
postgresql['trust_auth_cidr_addresses'] = ['127.0.0.1/32','1.2.3.4/32']
postgresql['md5_auth_cidr_addresses'] = ['5.6.7.8/32']
postgresql['sql_replication_user'] = "gitlab_replicator"
postgresql['wal_level'] = "hot_standby"
postgresql['max_wal_senders'] = 10
postgresql['wal_keep_segments'] = 10
postgresql['hot_standby'] = "on"
# postgresql['max_wal_senders'] = 10
# postgresql['wal_keep_segments'] = 10
```
Where `1.2.3.4` is the public IP address of the primary server, and `5.6.7.8`
......@@ -109,7 +107,9 @@ The following guide assumes that:
postgresql['md5_auth_cidr_addresses'] = ['5.6.7.8/32','11.22.33.44/32']
```
Edit the `wal` values as you see fit.
You may also want to edit the `wal_keep_segments` and `max_wal_senders` to
match your database replication requirements. Consult the [PostgreSQL - Replication documentation](https://www.postgresql.org/docs/9.6/static/runtime-config-replication.html)
for more information.
1. Check to make sure your firewall rules are set so that the secondary nodes
can access port 5432 on the primary node.
......@@ -146,11 +146,7 @@ The following guide assumes that:
1. Edit `/etc/gitlab/gitlab.rb` and add the following:
```ruby
postgresql['wal_level'] = "hot_standby"
postgresql['max_wal_senders'] = 10
postgresql['wal_keep_segments'] = 10
postgresql['hot_standby'] = "on"
gitlab_rails['auto_migrate'] = false # prevents migrations to be executed on the secondary server
geo_secondary_role['enable'] = true
```
1. [Reconfigure GitLab][] for the changes to take effect.
......@@ -175,59 +171,15 @@ data before running `pg_basebackup`.
sudo -i
```
1. Save the snippet below in a file, let's say `/tmp/replica.sh`:
```bash
#!/bin/bash
PORT="5432"
USER="gitlab_replicator"
echo ---------------------------------------------------------------
echo WARNING: Make sure this scirpt is run from the secondary server
echo ---------------------------------------------------------------
echo
echo Enter the IP of the primary PostgreSQL server
read HOST
echo Enter the password for $USER@$HOST
read -s PASSWORD
echo Stopping PostgreSQL and all GitLab services
gitlab-ctl stop
echo Backing up postgresql.conf
sudo -u gitlab-psql mv /var/opt/gitlab/postgresql/data/postgresql.conf /var/opt/gitlab/postgresql/
echo Cleaning up old cluster directory
sudo -u gitlab-psql rm -rf /var/opt/gitlab/postgresql/data
rm -f /tmp/postgresql.trigger
echo Starting base backup as the replicator user
echo Enter the password for $USER@$HOST
sudo -u gitlab-psql /opt/gitlab/embedded/bin/pg_basebackup -h $HOST -D /var/opt/gitlab/postgresql/data -U gitlab_replicator -v -x -P
echo Writing recovery.conf file
sudo -u gitlab-psql bash -c "cat > /var/opt/gitlab/postgresql/data/recovery.conf <<- _EOF1_
standby_mode = 'on'
primary_conninfo = 'host=$HOST port=$PORT user=$USER password=$PASSWORD'
trigger_file = '/tmp/postgresql.trigger'
_EOF1_
"
echo Restoring postgresql.conf
sudo -u gitlab-psql mv /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/data/
echo Starting PostgreSQL and all GitLab services
gitlab-ctl start
```
1. Run it with:
1. Execute the command below to start a backup/restore and begin the replication:
```
bash /tmp/replica.sh
gitlab-ctl replicate-geo-database --host=1.2.3.4
```
When prompted, enter the password you set up for the `gitlab_replicator`
user in the first step.
Change the `--host=` to the primary node IP or FQDN. You can check other possible
parameters with `--help`. When prompted, enter the password you set up for
the `gitlab_replicator` user in the first step.
The replication process is now over.
......
doc/gitlab-geo/database_source.md
View file @ b3288eec
......@@ -74,7 +74,9 @@ The following guide assumes that:
See the Omnibus notes above for more details of `listen_address`.
Edit the `wal` values as you see fit.
You may also want to edit the `wal_keep_segments` and `max_wal_senders` to
match your database replication requirements. Consult the [PostgreSQL - Replication documentation](https://www.postgresql.org/docs/9.6/static/runtime-config-replication.html)
for more information.
1. Set the access control on the primary to allow TCP connections using the
server's public IP and set the connection from the secondary to require a
......
lib/tasks/geo.rake
View file @ b3288eec
......@@ -42,6 +42,11 @@ namespace :geo do
Rake::Task['geo:config:restore'].reenable
end
end
desc 'Display database encryption key'
task show_encryption_key: :environment do
puts Rails.application.secrets.db_key_base
end
end
namespace :config do
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7