Skip to content

G
gitlab-ce
Commit c6cc15be authored by Achilleas Pipinellis's avatar Achilleas Pipinellis
Browse files
Options

Merge branch '4052-geo-doc-improvements' into 'master' 

Geo documentation fixes following on from 2017-11-14 demo

Closes #4052

See merge request gitlab-org/gitlab-ee!3410
parents 27022232 2a63675c
Show whitespace changes
Inline Side-by-side
Showing with 115 additions and 137 deletions
doc/gitlab-geo/configuration.md
View file @ c6cc15be
......@@ -16,7 +16,7 @@ Before attempting the steps in this stage, complete all prior stages.
1. [Setup the database replication](database.md) (`primary (read-write) <-> secondary (read-only)` topology).
1. [Configure SSH authorizations to use the database](ssh.md)
1. **Configure GitLab to set the primary and secondary nodes.**
1. Optional: [Configure a secondary LDAP server](../administration/auth/ldap.md) for the secondary. See [notes on LDAP](#ldap).
1. Optional: [Configure a secondary LDAP server](../administration/auth/ldap.md) for the secondary.
1. [Follow the after setup steps](after_setup.md).
[install-ee]: https://about.gitlab.com/downloads-ee/ "GitLab Enterprise Edition Omnibus packages downloads page"
......@@ -38,16 +38,14 @@ After having installed GitLab Enterprise Edition in the instance that will serve
as a Geo node and set up the [database replication](database.md), the next steps
can be summed up to:
1. Configure the primary node
1. Replicate some required configurations between the primary and the secondaries
1. Configure a second, tracking database on each secondary
1. Configure every secondary node in the primary's Admin screen
1. Start GitLab on the secondary node's machine
### Prerequisites
This is the last step of configuring a Geo node. Make sure you have followed the
first two steps of the [Setup instructions](README.md#setup-instructions):
This is the last step of configuring a Geo secondary node. Make sure you have
followed the first two steps of the [Setup instructions](README.md#setup-instructions):
1. You have already installed on the secondary server the same version of
GitLab Enterprise Edition that is present on the primary server.
......@@ -57,31 +55,8 @@ first two steps of the [Setup instructions](README.md#setup-instructions):
1. Your nodes must have an NTP service running to synchronize the clocks.
You can use different timezones, but the hour relative to UTC can't be more
than 60 seconds off from each node.
1. You have set up another PostgreSQL database that can store writes for the secondary.
Note that this MUST be on another instance, since the primary replicated database
is read-only.
Some of the following steps require to configure the primary and secondary
nodes almost at the same time. For your convenience make sure you have SSH
logins opened on all nodes as we will be moving back and forth.
### Step 1. Adding the primary GitLab node
1. SSH into the **primary** node and login as root:
```
sudo -i
```
1. Execute the command below to define the node as primary Geo node:
```
gitlab-ctl set-geo-primary-node
```
This command will use your defined `external_url` in `gitlab.rb`
### Step 2. Copying the database encryption key
### Step 1. Copying the database encryption key
GitLab stores a unique encryption key in disk that we use to safely store
sensitive data in the database. Any secondary node must have the
......@@ -105,16 +80,15 @@ sensitive data in the database. Any secondary node must have the
sudo -i
```
1. Open the secrets file and paste the value of `db_key_base` you copied in the
previous step:
1. Add the following to /etc/gitlab/gitlab.rb, replacing `encryption-key` with the output
of the previous command:
```
editor /etc/gitlab/gitlab-secrets.json
```ruby
gitlab_rails['db_key_base'] = "encryption-key"
```
1. Save and close the file.
1. Reconfigure the secondary node for the change to take effect:
1. Reconfigure for the change to take effect.
```
gitlab-ctl reconfigure
```
......@@ -125,7 +99,7 @@ Meanwhile, the primary node will start to notify changes to the secondary, which
will act on those notifications immediately. Make sure the secondary instance is
running and accessible.
### Step 3. Enabling hashed storage (from GitLab 10.0)
### Step 2. Enabling hashed storage (from GitLab 10.0)
1. Visit the **primary** node's **Admin Area ➔ Settings**
(`/admin/application_settings`) in your browser
......@@ -137,7 +111,7 @@ Using hashed storage significantly improves Geo replication - project and group
renames no longer require synchronization between nodes - so we recommend it is
used for all GitLab Geo installations.
### Step 4. (Optional) Configuring the secondary to trust the primary
### Step 3. (Optional) Configuring the secondary to trust the primary
You can safely skip this step if your primary uses a CA-issued HTTPS certificate.
......@@ -147,7 +121,7 @@ certificate from the primary and follow
[these instructions](https://docs.gitlab.com/omnibus/settings/ssl.html)
on the secondary.
### Step 5. Managing the secondary GitLab node
### Step 4. Managing the secondary GitLab node
You can monitor the status of the syncing process on a secondary node
by visiting the primary node's **Admin Area ➔ Geo Nodes** (`/admin/geo_nodes`)
......@@ -204,12 +178,6 @@ secondary nodes, but repositories that have not been selected will be empty.
1. Secondary nodes won't pull repositories that do not belong to the selected
groups to be replicated.
## Adding another secondary Geo node
To add another Geo node in an already Geo configured infrastructure, just follow
[the steps starting from step 2](#step-2-copying-the-database-encryption-key)
Just omit the first step that sets up the primary node.
## Replicating wikis and repositories over SSH
In GitLab 10.2, replicating repositories and wikis over SSH was deprecated.
......@@ -241,7 +209,7 @@ to add a new secondary in the short term, you can follow these instructions:
```
Follow the steps above to set up the new Geo node. When you reach
[Step 5: Enabling the secondary GitLab node](#step-5-managing-the-secondary-gitlab-node)
[Step 4: Enabling the secondary GitLab node](#step-4-managing-the-secondary-gitlab-node)
select "SSH (deprecated)" instead of "HTTP/HTTPS", and populate the "Public Key"
with the output of the previous command (beginning `ssh-rsa AAAA...`).
......
doc/gitlab-geo/configuration_source.md
View file @ c6cc15be
......@@ -37,16 +37,14 @@ After having installed GitLab Enterprise Edition in the instance that will serve
as a Geo node and set up the [database replication](database_source.md), the
next steps can be summed up to:
1. Configure the primary node
1. Replicate some required configurations between the primary and the secondaries
1. Configure a second, tracking database on each secondary
1. Configure every secondary node in the primary's Admin screen
1. Start GitLab on the secondary node's machine
### Prerequisites
This is the last step of configuring a Geo node. Make sure you have followed the
first two steps of the [Setup instructions](README.md#setup-instructions):
This is the last step of configuring a Geo secondary node. Make sure you have
followed the first two steps of the [Setup instructions](README.md#setup-instructions):
1. You have already installed on the secondary server the same version of
GitLab Enterprise Edition that is present on the primary server.
......@@ -60,25 +58,7 @@ first two steps of the [Setup instructions](README.md#setup-instructions):
Note that this MUST be on another instance, since the primary replicated database
is read-only.
Some of the following steps require to configure the primary and secondary
nodes almost at the same time. For your convenience make sure you have SSH
logins opened on all nodes as we will be moving back and forth.
### Step 1. Adding the primary GitLab node
1. SSH into the **primary** node and login as root:
```
sudo -i
```
1. Add this node as the Geo primary by running:
```bash
bundle exec rake geo:set_primary_node
```
### Step 2. Copying the database encryption key
### Step 1. Copying the database encryption key
GitLab stores a unique encryption key in disk that we use to safely store
sensitive data in the database. Any secondary node must have the
......@@ -93,7 +73,7 @@ sensitive data in the database. Any secondary node must have the
1. Execute the command below to display the current encryption key and copy it:
```
bundle exec rake geo:db:show_encryption_key
sudo -u git -H bundle exec rake geo:db:show_encryption_key RAILS_ENV=production
```
1. SSH into the **secondary** node and login as root:
......@@ -102,21 +82,27 @@ sensitive data in the database. Any secondary node must have the
sudo -i
```
1. Open the secrets file and paste the value of `db_key_base` you copied in the
previous step:
1. Open the `secrets.yml` file and change the value of `db_key_base` to the
output of the previous step:
```
editor /etc/gitlab/gitlab-secrets.json
sudo -u git -H editor config/secrets.yml
```
1. Save and close the file.
1. Restart GitLab for the changes to take effect:
```
service gitlab restart
```
The secondary will start automatically replicating missing data from the
primary in a process known as backfill. Meanwhile, the primary node will start
to notify changes to the secondary, which will act on those notifications
immediately. Make sure the secondary instance is running and accessible.
### Step 3. Enabling hashed storage (from GitLab 10.0)
### Step 2. Enabling hashed storage (from GitLab 10.0)
1. Visit the **primary** node's **Admin Area ➔ Settings**
(`/admin/application_settings`) in your browser
......@@ -128,7 +114,7 @@ Using hashed storage significantly improves Geo replication - project and group
renames no longer require synchronization between nodes - so we recommend it is
used for all GitLab Geo installations.
### Step 4. (Optional) Configuring the secondary to trust the primary
### Step 3. (Optional) Configuring the secondary to trust the primary
You can safely skip this step if your primary uses a CA-issued HTTPS certificate.
......@@ -144,7 +130,7 @@ cp primary.geo.example.com.crt /usr/local/share/ca-certificates
update-ca-certificates
```
### Step 5. Managing the secondary GitLab node
### Step 4. Managing the secondary GitLab node
You can monitor the status of the syncing process on a secondary node
by visiting the primary node's **Admin Area ➔ Geo Nodes** (`/admin/geo_nodes`)
......@@ -194,12 +180,6 @@ Point your users to the [after setup steps](after_setup.md).
Read [Selective replication](configuration.md#selective-replication).
## Adding another secondary Geo node
To add another Geo node in an already Geo configured infrastructure, just follow
[the steps starting from step 2](#step-2-copying-the-database-encryption-key).
Just omit the first step that sets up the primary node.
## Replicating wikis and repositories over SSH
Read [Replicating wikis and repositories over SSH](configuration.md#replicating-wikis-and-repositories-over-ssh).
......
doc/gitlab-geo/database.md
View file @ c6cc15be
......@@ -67,6 +67,14 @@ The following guide assumes that:
sudo -i
```
1. Execute the command below to define the node as primary Geo node:
```
gitlab-ctl set-geo-primary-node
```
This command will use your defined `external_url` in `/etc/gitlab/gitlab.rb`.
1. Omnibus GitLab has already a replication user called `gitlab_replicator`.
You must set its password manually. You will be prompted to enter a
password:
......@@ -248,10 +256,44 @@ primary before the database is replicated.
sudo -i
```
1. Test that the remote connection to the primary server works:
1. Set up PostgreSQL TLS verification on the secondary
If you configured PostgreSQL to accept TLS connections in
[Step 1](#step-1-configure-the-primary-server), then you need to provide a
list of "known-good" certificates to the secondary. It uses this list to
keep the connection secure against an active "man-in-the-middle" attack.
If you reused your existing certificates on the primary, you can use the
list of valid root certificates provided with omnibus:
```bash
mkdir -p ~gitlab-psql/.postgresql
ln -s /opt/gitlab/embedded/ssl/certs/cacert.pem ~gitlab-psql/.postgresql/root.crt
```
sudo -u gitlab-psql /opt/gitlab/embedded/bin/psql -h 1.2.3.4 -U gitlab_replicator -d gitlabhq_production -W
If you generated a self-signed certificate, that won't work. Copy the
generated `server.crt` file onto the secondary server from the primary, then
install it in the right place:
```bash
install -o gitlab-psql -g gitlab-psql -m 0400 -T server.crt ~gitlab-psql/.postgresql/root.crt
```
PostgreSQL will now only recognize that exact certificate when verifying TLS
connections.
1. Test that the remote connection to the primary server works.
If you're using a CA-issued certificate and connecting by FQDN:
```
sudo -u gitlab-psql /opt/gitlab/embedded/bin/psql -h primary.geo.example.com -U gitlab_replicator -d "dbname=gitlabhq_production sslmode=verify-ca" -W
```
If you're using a self-signed certificate or connecting by IP address:
```
sudo -u gitlab-psql /opt/gitlab/embedded/bin/psql -h 1.2.3.4 -U gitlab_replicator -d "dbname=gitlabhq_production sslmode=verify-full" -W
```
When prompted enter the password you set in the first step for the
......@@ -305,31 +347,6 @@ data before running `pg_basebackup`.
`geo-secondary.mydomain.com`, you may use `geo_secondary_my_domain_com` as
the slot name.
1. Set up PostgreSQL TLS verification on the secondary
If you configured the PostgreSQL to accept TLS connections in
[Step 1][#step-1-configure-the-primary-server], then you need to provide a
list of "known-good" certificates to the secondary. It uses this list to
keep the connection secure against an active "man-in-the-middle" attack.
If you reused your existing certificates on the primary, you can use the
list of valid root certificates provided with omnibus:
```bash
mkdir -p ~gitlab-psql/.postgresql
ln -s /opt/gitlab/embedded/ssl/certs/cacert.pem ~gitlab-psql/.postgresql/root.crt
```
If you generated a self-signed certificate, that won't work. Copy the
generated `server.crt` file onto the secondary server from the primary, then
install it in the right place:
```bash
install -o gitlab-psql -g gitlab-psql -m 0400 -T server.crt ~gitlab-psql/.postgresql/root.crt
```
PostgreSQL will now only recognize that exact certificate when verifying TLS
connections.
1. Execute the command below to start a backup/restore and begin the replication:
```
......
doc/gitlab-geo/database_source.md
View file @ c6cc15be
......@@ -64,6 +64,12 @@ The following guide assumes that:
sudo -i
```
1. Add this node as the Geo primary by running:
```bash
bundle exec rake geo:set_primary_node
```
1. Create a replication user named `gitlab_replicator`:
```bash
......@@ -231,10 +237,44 @@ primary before the database is replicated.
sudo -i
```
1. Set up PostgreSQL TLS verification on the secondary
If you configured PostgreSQL to accept TLS connections in
[Step 1](#step-1-configure-the-primary-server), then you need to provide a
list of "known-good" certificates to the secondary. It uses this list to
keep the connection secure against an active "man-in-the-middle" attack.
If you reused your existing certificates on the primary, you can use the
list of valid root certificates provided with your distribution. For
Debian/Ubuntu, they can be found in `/etc/ssl/certs/ca-certificates.crt`:
```bash
mkdir -p ~postgres/.postgresql
ln -s /etc/ssl/certs/ca-certificates.crt ~postgres/.postgresql/root.crt
```
If you generated a self-signed certificate, that won't work. Copy the
generated `server.crt` file onto the secondary server from the primary, then
install it in the right place:
```bash
install -o postgres -g postgres -m 0400 -T server.crt ~postgres/.postgresql/root.crt
```
PostgreSQL will now only recognize that exact certificate when verifying TLS
connections.
1. Test that the remote connection to the primary server works:
If you're using a CA-issued certificate and connecting by FQDN:
```
sudo -u postgres psql -h 1.2.3.4 -U gitlab_replicator -d gitlabhq_production -W
sudo -u postgres psql -h primary.geo.example.com -U gitlab_replicator -d "dbname=gitlabhq_production sslmode=verify-ca" -W
```
If you're using a self-signed certificate or connecting by IP address:
```
sudo -u postgres psql -h 1.2.3.4 -U gitlab_replicator -d "dbname=gitlabhq_production sslmode=verify-full" -W
```
When prompted enter the password you set in the first step for the
......@@ -333,33 +373,6 @@ data before running `pg_basebackup`.
sudo -i
```
1. Set up PostgreSQL TLS verification on the secondary
If you configured the PostgreSQL to accept TLS connections in
[Step 1][#step-1-configure-the-primary-server], then you need to provide a
list of "known-good" certificates to the secondary. It uses this list to
keep the connection secure against an active "man-in-the-middle" attack.
If you reused your existing certificates on the primary, you can use the
list of valid root certificates provided with your distribution. For
Debian/Ubuntu, they can be found in `/etc/ssl/certs/ca-certificates.crt`:
```bash
mkdir -p ~postgres/.postgresql
ln -s /etc/ssl/certs/ca-certificates.crt ~postgres/.postgresql/root.crt
```
If you generated a self-signed certificate, that won't work. Copy the
generated `server.crt` file onto the secondary server from the primary, then
install it in the right place:
```bash
install -o postgres -g postgres -m 0400 -T server.crt ~postgres/.postgresql/root.crt
```
PostgreSQL will now only recognize that exact certificate when verifying TLS
connections.
1. Save the snippet below in a file, let's say `/tmp/replica.sh`. Modify the
embedded paths if necessary:
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7