Skip to content

G
gitlab-ce
Commit 3b88f335 authored by Sunjung Park's avatar Sunjung Park Committed by Kati Paizee
Browse files
Options

Replace Geo node to site in the replication docs

parent ad1dbf5b
Hide whitespace changes
Inline Side-by-side
Showing with 79 additions and 79 deletions
doc/administration/geo/replication/configuration.md
View file @ 3b88f335
...@@ -7,35 +7,35 @@ type: howto ...@@ -7,35 +7,35 @@ type: howto
# Geo configuration **(PREMIUM SELF)** # Geo configuration **(PREMIUM SELF)**
## Configuring a new **secondary** node ## Configuring a new **secondary** site
NOTE: NOTE:
This is the final step in setting up a **secondary** Geo node. Stages of the This is the final step in setting up a **secondary** Geo site. Stages of the
setup process must be completed in the documented order. setup process must be completed in the documented order.
Before attempting the steps in this stage, [complete all prior stages](../setup/index.md#using-omnibus-gitlab). Before attempting the steps in this stage, [complete all prior stages](../setup/index.md#using-omnibus-gitlab).
The basic steps of configuring a **secondary** node are to: The basic steps of configuring a **secondary** site are to:
- Replicate required configurations between the **primary** node and the **secondary** nodes. - Replicate required configurations between the **primary** site and the **secondary** sites.
- Configure a tracking database on each **secondary** node. - Configure a tracking database on each **secondary** site.
- Start GitLab on each **secondary** node. - Start GitLab on each **secondary** site.
You are encouraged to first read through all the steps before executing them You are encouraged to first read through all the steps before executing them
in your testing/production environment. in your testing/production environment.
NOTE: NOTE:
**Do not** set up any custom authentication for the **secondary** nodes. This is handled by the **primary** node. **Do not** set up any custom authentication for the **secondary** sites. This is handled by the **primary** site.
Any change that requires access to the **Admin Area** needs to be done in the Any change that requires access to the **Admin Area** needs to be done in the
**primary** node because the **secondary** node is a read-only replica. **primary** site because the **secondary** site is a read-only replica.
### Step 1. Manually replicate secret GitLab values ### Step 1. Manually replicate secret GitLab values
GitLab stores a number of secret values in the `/etc/gitlab/gitlab-secrets.json` GitLab stores a number of secret values in the `/etc/gitlab/gitlab-secrets.json`
file which *must* be the same on all nodes. Until there is file which *must* be the same on all of a site's nodes. Until there is
a means of automatically replicating these between nodes (see [issue #3789](https://gitlab.com/gitlab-org/gitlab/-/issues/3789)), a means of automatically replicating these between sites (see [issue #3789](https://gitlab.com/gitlab-org/gitlab/-/issues/3789)),
they must be manually replicated to the **secondary** node. they must be manually replicated to **all nodes of the secondary site**.
1. SSH into the **primary** node, and execute the command below: 1. SSH into a **Rails node on your primary** site, and execute the command below:
```shell ```shell
sudo cat /etc/gitlab/gitlab-secrets.json sudo cat /etc/gitlab/gitlab-secrets.json
...@@ -43,7 +43,7 @@ they must be manually replicated to the **secondary** node. ...@@ -43,7 +43,7 @@ they must be manually replicated to the **secondary** node.
This displays the secrets that need to be replicated, in JSON format. This displays the secrets that need to be replicated, in JSON format.
1. SSH into the **secondary** node and login as the `root` user: 1. SSH **into each node on your secondary Geo site** and login as the `root` user:
```shell ```shell
sudo -i sudo -i
...@@ -55,7 +55,7 @@ they must be manually replicated to the **secondary** node. ...@@ -55,7 +55,7 @@ they must be manually replicated to the **secondary** node.
mv /etc/gitlab/gitlab-secrets.json /etc/gitlab/gitlab-secrets.json.`date +%F` mv /etc/gitlab/gitlab-secrets.json /etc/gitlab/gitlab-secrets.json.`date +%F`
``` ```
1. Copy `/etc/gitlab/gitlab-secrets.json` from the **primary** node to the **secondary** node, or 1. Copy `/etc/gitlab/gitlab-secrets.json` from the **Rails node on your primary** site to **each node on your secondary** site, or
copy-and-paste the file contents between nodes: copy-and-paste the file contents between nodes:
```shell ```shell
...@@ -72,28 +72,28 @@ they must be manually replicated to the **secondary** node. ...@@ -72,28 +72,28 @@ they must be manually replicated to the **secondary** node.
chmod 0600 /etc/gitlab/gitlab-secrets.json chmod 0600 /etc/gitlab/gitlab-secrets.json
``` ```
1. Reconfigure the **secondary** node for the change to take effect: 1. Reconfigure **each Rails, Sidekiq and Gitaly nodes on your secondary** site for the change to take effect:
```shell ```shell
gitlab-ctl reconfigure gitlab-ctl reconfigure
gitlab-ctl restart gitlab-ctl restart
``` ```
### Step 2. Manually replicate the **primary** node's SSH host keys ### Step 2. Manually replicate the **primary** site's SSH host keys
GitLab integrates with the system-installed SSH daemon, designating a user GitLab integrates with the system-installed SSH daemon, designating a user
(typically named `git`) through which all access requests are handled. (typically named `git`) through which all access requests are handled.
In a [Disaster Recovery](../disaster_recovery/index.md) situation, GitLab system In a [Disaster Recovery](../disaster_recovery/index.md) situation, GitLab system
administrators promote a **secondary** node to the **primary** node. DNS records for the administrators promote a **secondary** site to the **primary** site. DNS records for the
**primary** domain should also be updated to point to the new **primary** node **primary** domain should also be updated to point to the new **primary** site
(previously a **secondary** node). Doing so avoids the need to update Git remotes and API URLs. (previously a **secondary** site). Doing so avoids the need to update Git remotes and API URLs.
This causes all SSH requests to the newly promoted **primary** node to This causes all SSH requests to the newly promoted **primary** site to
fail due to SSH host key mismatch. To prevent this, the primary SSH host fail due to SSH host key mismatch. To prevent this, the primary SSH host
keys must be manually replicated to the **secondary** node. keys must be manually replicated to the **secondary** site.
1. SSH into the **secondary** node and login as the `root` user: 1. SSH into **each node on your secondary** site and login as the `root` user:
```shell ```shell
sudo -i sudo -i
...@@ -105,34 +105,34 @@ keys must be manually replicated to the **secondary** node. ...@@ -105,34 +105,34 @@ keys must be manually replicated to the **secondary** node.
find /etc/ssh -iname ssh_host_* -exec cp {} {}.backup.`date +%F` \; find /etc/ssh -iname ssh_host_* -exec cp {} {}.backup.`date +%F` \;
``` ```
1. Copy OpenSSH host keys from the **primary** node: 1. Copy OpenSSH host keys from the **primary** site:
If you can access your **primary** node using the **root** user: If you can access one of the **nodes on your primary** site serving SSH traffic (usually, the main GitLab Rails application nodes) using the **root** user:
```shell ```shell
# Run this from the secondary node, change `<primary_node_fqdn>` for the IP or FQDN of the server # Run this from the secondary site, change `<primary_site_fqdn>` for the IP or FQDN of the server
scp root@<primary_node_fqdn>:/etc/ssh/ssh_host_*_key* /etc/ssh scp root@<primary_node_fqdn>:/etc/ssh/ssh_host_*_key* /etc/ssh
``` ```
If you only have access through a user with `sudo` privileges: If you only have access through a user with `sudo` privileges:
```shell ```shell
# Run this from your primary node: # Run this from the node on your primary site:
sudo tar --transform 's/.*\///g' -zcvf ~/geo-host-key.tar.gz /etc/ssh/ssh_host_*_key* sudo tar --transform 's/.*\///g' -zcvf ~/geo-host-key.tar.gz /etc/ssh/ssh_host_*_key*
# Run this from your secondary node: # Run this on each node on your secondary site:
scp <user_with_sudo>@<primary_node_fqdn>:geo-host-key.tar.gz . scp <user_with_sudo>@<primary_site_fqdn>:geo-host-key.tar.gz .
tar zxvf ~/geo-host-key.tar.gz -C /etc/ssh tar zxvf ~/geo-host-key.tar.gz -C /etc/ssh
``` ```
1. On your **secondary** node, ensure the file permissions are correct: 1. On **each node on your secondary** site, ensure the file permissions are correct:
```shell ```shell
chown root:root /etc/ssh/ssh_host_*_key* chown root:root /etc/ssh/ssh_host_*_key*
chmod 0600 /etc/ssh/ssh_host_*_key* chmod 0600 /etc/ssh/ssh_host_*_key*
``` ```
1. To verify key fingerprint matches, execute the following command on both nodes: 1. To verify key fingerprint matches, execute the following command on both primary and secondary nodes on each site:
```shell ```shell
for file in /etc/ssh/ssh_host_*_key; do ssh-keygen -lf $file; done for file in /etc/ssh/ssh_host_*_key; do ssh-keygen -lf $file; done
...@@ -160,7 +160,7 @@ keys must be manually replicated to the **secondary** node. ...@@ -160,7 +160,7 @@ keys must be manually replicated to the **secondary** node.
NOTE: NOTE:
The output for private keys and public keys command should generate the same fingerprint. The output for private keys and public keys command should generate the same fingerprint.
1. Restart `sshd` on your **secondary** node: 1. Restart `sshd` on **each node on your secondary** site:
```shell ```shell
# Debian or Ubuntu installations # Debian or Ubuntu installations
...@@ -175,31 +175,31 @@ keys must be manually replicated to the **secondary** node. ...@@ -175,31 +175,31 @@ keys must be manually replicated to the **secondary** node.
SSH into your GitLab **secondary** server in a new terminal. If you are unable to connect, SSH into your GitLab **secondary** server in a new terminal. If you are unable to connect,
verify the permissions are correct according to the previous steps. verify the permissions are correct according to the previous steps.
### Step 3. Add the **secondary** node ### Step 3. Add the **secondary** site
1. SSH into your GitLab **secondary** server and login as root: 1. SSH into **each Rails and Sidekiq node on your secondary** site and login as root:
```shell ```shell
sudo -i sudo -i
``` ```
1. Edit `/etc/gitlab/gitlab.rb` and add a **unique** name for your node. You need this in the next steps: 1. Edit `/etc/gitlab/gitlab.rb` and add a **unique** name for your site. You need this in the next steps:
```ruby ```ruby
# The unique identifier for the Geo node. # The unique identifier for the Geo site.
gitlab_rails['geo_node_name'] = '<node_name_here>' gitlab_rails['geo_node_name'] = '<site_name_here>'
``` ```
1. Reconfigure the **secondary** node for the change to take effect: 1. Reconfigure **each Rails and Sidekiq node on your secondary** site for the change to take effect:
```shell ```shell
gitlab-ctl reconfigure gitlab-ctl reconfigure
``` ```
1. On the top bar of the primary node, select **Menu >** **{admin}** **Admin**. 1. On the top bar, select **Menu >** **{admin}** **Admin**.
1. On the left sidebar, select **Geo > Nodes**. 1. On the left sidebar, select **Geo > Sites**.
1. Select **Add site**. 1. Select **New site**.
![Add secondary node](img/adding_a_secondary_node_v13_3.png) ![Add secondary site](img/adding_a_secondary_node_v13_3.png)
1. Fill in **Name** with the `gitlab_rails['geo_node_name']` in 1. Fill in **Name** with the `gitlab_rails['geo_node_name']` in
`/etc/gitlab/gitlab.rb`. These values must always match *exactly*, character `/etc/gitlab/gitlab.rb`. These values must always match *exactly*, character
for character. for character.
...@@ -207,10 +207,10 @@ keys must be manually replicated to the **secondary** node. ...@@ -207,10 +207,10 @@ keys must be manually replicated to the **secondary** node.
values must always match, but it doesn't matter if one ends with a `/` and values must always match, but it doesn't matter if one ends with a `/` and
the other doesn't. the other doesn't.
1. Optionally, choose which groups or storage shards should be replicated by the 1. Optionally, choose which groups or storage shards should be replicated by the
**secondary** node. Leave blank to replicate all. Read more in **secondary** site. Leave blank to replicate all. Read more in
[selective synchronization](#selective-synchronization). [selective synchronization](#selective-synchronization).
1. Select **Add node** to add the **secondary** node. 1. Select **Add site** to add the **secondary** site.
1. SSH into your GitLab **secondary** server and restart the services: 1. SSH into **each Rails, and Sidekiq node on your secondary** site and restart the services:
```shell ```shell
gitlab-ctl restart gitlab-ctl restart
...@@ -222,56 +222,56 @@ keys must be manually replicated to the **secondary** node. ...@@ -222,56 +222,56 @@ keys must be manually replicated to the **secondary** node.
gitlab-rake gitlab:geo:check gitlab-rake gitlab:geo:check
``` ```
1. SSH into your **primary** server and login as root to verify the 1. SSH into a **Rails or Sidekiq server on your primary** site and login as root to verify the
**secondary** node is reachable or there are any common issue with your Geo setup: **secondary** site is reachable or there are any common issue with your Geo setup:
```shell ```shell
gitlab-rake gitlab:geo:check gitlab-rake gitlab:geo:check
``` ```
Once added to the Geo administration page and restarted, the **secondary** node automatically starts Once added to the Geo administration page and restarted, the **secondary** site automatically starts
replicating missing data from the **primary** node in a process known as **backfill**. replicating missing data from the **primary** site in a process known as **backfill**.
Meanwhile, the **primary** node starts to notify each **secondary** node of any changes, so Meanwhile, the **primary** site starts to notify each **secondary** site of any changes, so
that the **secondary** node can act on those notifications immediately. that the **secondary** site can act on those notifications immediately.
Be sure the _secondary_ node is running and accessible. You can sign in to the Be sure the _secondary_ site is running and accessible. You can sign in to the
_secondary_ node with the same credentials as were used with the _primary_ node. _secondary_ site with the same credentials as were used with the _primary_ site.
### Step 4. (Optional) Configuring the **secondary** node to trust the **primary** node ### Step 4. (Optional) Configuring the **secondary** site to trust the **primary** site
You can safely skip this step if your **primary** node uses a CA-issued HTTPS certificate. You can safely skip this step if your **primary** site uses a CA-issued HTTPS certificate.
If your **primary** node is using a self-signed certificate for *HTTPS* support, you If your **primary** site is using a self-signed certificate for *HTTPS* support, you
need to add that certificate to the **secondary** node's trust store. Retrieve the need to add that certificate to the **secondary** site's trust store. Retrieve the
certificate from the **primary** node and follow certificate from the **primary** site and follow
[these instructions](https://docs.gitlab.com/omnibus/settings/ssl.html) [these instructions](https://docs.gitlab.com/omnibus/settings/ssl.html)
on the **secondary** node. on the **secondary** site.
### Step 5. Enable Git access over HTTP/HTTPS ### Step 5. Enable Git access over HTTP/HTTPS
Geo synchronizes repositories over HTTP/HTTPS, and therefore requires this clone Geo synchronizes repositories over HTTP/HTTPS, and therefore requires this clone
method to be enabled. This is enabled by default, but if converting an existing node to Geo it should be checked: method to be enabled. This is enabled by default, but if converting an existing site to Geo it should be checked:
On the **primary** node: On the **primary** site:
1. On the top bar, select **Menu >** **{admin}** **Admin**. 1. On the top bar, select **Menu >** **{admin}** **Admin**.
1. On the left sidebar, select **Settings > General**. 1. On the left sidebar, select **Settings > General**.
1. Expand **Visibility and access controls**. 1. Expand **Visibility and access controls**.
1. Ensure "Enabled Git access protocols" is set to either "Both SSH and HTTP(S)" or "Only HTTP(S)". 1. Ensure "Enabled Git access protocols" is set to either "Both SSH and HTTP(S)" or "Only HTTP(S)".
### Step 6. Verify proper functioning of the **secondary** node ### Step 6. Verify proper functioning of the **secondary** site
You can sign in to the **secondary** node with the same credentials you used with You can sign in to the **secondary** site with the same credentials you used with
the **primary** node. After you sign in: the **primary** site. After you sign in:
1. On the top bar, select **Menu >** **{admin}** **Admin**. 1. On the top bar, select **Menu >** **{admin}** **Admin**.
1. On the left sidebar, select **Geo > Nodes**. 1. On the left sidebar, select **Geo > Sites**.
1. Verify that it's correctly identified as a **secondary** Geo node, and that 1. Verify that it's correctly identified as a **secondary** Geo site, and that
Geo is enabled. Geo is enabled.
The initial replication, or 'backfill', is probably still in progress. You The initial replication, or 'backfill', is probably still in progress. You
can monitor the synchronization process on each Geo node from the **primary** can monitor the synchronization process on each Geo site from the **primary**
node's **Geo Nodes** dashboard in your browser. site's **Geo Sites** dashboard in your browser.
![Geo dashboard](img/geo_node_dashboard_v14_0.png) ![Geo dashboard](img/geo_node_dashboard_v14_0.png)
...@@ -286,10 +286,10 @@ The two most obvious issues that can become apparent in the dashboard are: ...@@ -286,10 +286,10 @@ The two most obvious issues that can become apparent in the dashboard are:
- You are using a custom certificate or custom CA (see the [troubleshooting document](troubleshooting.md)). - You are using a custom certificate or custom CA (see the [troubleshooting document](troubleshooting.md)).
- The instance is firewalled (check your firewall rules). - The instance is firewalled (check your firewall rules).
Disabling a **secondary** node stops the synchronization process. Please note that disabling a **secondary** site stops the synchronization process.
If `git_data_dirs` is customized on the **primary** node for multiple Please note that if `git_data_dirs` is customized on the **primary** site for multiple
repository shards you must duplicate the same configuration on each **secondary** node. repository shards you must duplicate the same configuration on each **secondary** site.
Point your users to the [Using a Geo Site guide](usage.md). Point your users to the [Using a Geo Site guide](usage.md).
...@@ -304,7 +304,7 @@ Currently, this is what is synced: ...@@ -304,7 +304,7 @@ Currently, this is what is synced:
## Selective synchronization ## Selective synchronization
Geo supports selective synchronization, which allows administrators to choose Geo supports selective synchronization, which allows administrators to choose
which projects should be synchronized by **secondary** nodes. which projects should be synchronized by **secondary** sites.
A subset of projects can be chosen, either by group or by storage shard. The A subset of projects can be chosen, either by group or by storage shard. The
former is ideal for replicating data belonging to a subset of users, while the former is ideal for replicating data belonging to a subset of users, while the
latter is more suited to progressively rolling out Geo to a large GitLab latter is more suited to progressively rolling out Geo to a large GitLab
...@@ -312,22 +312,22 @@ instance. ...@@ -312,22 +312,22 @@ instance.
It is important to note that selective synchronization: It is important to note that selective synchronization:
1. Does not restrict permissions from **secondary** nodes. 1. Does not restrict permissions from **secondary** sites.
1. Does not hide project metadata from **secondary** nodes. 1. Does not hide project metadata from **secondary** sites.
- Since Geo currently relies on PostgreSQL replication, all project metadata - Since Geo currently relies on PostgreSQL replication, all project metadata
gets replicated to **secondary** nodes, but repositories that have not been gets replicated to **secondary** sites, but repositories that have not been
selected are empty. selected are empty.
1. Does not reduce the number of events generated for the Geo event log. 1. Does not reduce the number of events generated for the Geo event log.
- The **primary** node generates events as long as any **secondary** nodes are present. - The **primary** site generates events as long as any **secondary** sites are present.
Selective synchronization restrictions are implemented on the **secondary** nodes, Selective synchronization restrictions are implemented on the **secondary** sites,
not the **primary** node. not the **primary** site.
### Git operations on unreplicated repositories ### Git operations on unreplicated repositories
> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2562) in GitLab 12.10 for HTTP(S) and in GitLab 13.0 for SSH. > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2562) in GitLab 12.10 for HTTP(S) and in GitLab 13.0 for SSH.
Git clone, pull, and push operations over HTTP(S) and SSH are supported for repositories that Git clone, pull, and push operations over HTTP(S) and SSH are supported for repositories that
exist on the **primary** node but not on **secondary** nodes. This situation can occur exist on the **primary** site but not on **secondary** sites. This situation can occur
when: when:
- Selective synchronization does not include the project attached to the repository. - Selective synchronization does not include the project attached to the repository.
...@@ -335,7 +335,7 @@ when: ...@@ -335,7 +335,7 @@ when:
## Upgrading Geo ## Upgrading Geo
See the [updating the Geo nodes document](updating_the_geo_nodes.md). See the [updating the Geo sites document](updating_the_geo_nodes.md).
## Troubleshooting ## Troubleshooting
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7