Commit 8894b3c5 authored by Evan Read's avatar Evan Read

Merge branch 'docs-rnienaber-snippets-4' into 'master'

Changing long snippets into multiline and adding alligator brackets

See merge request gitlab-org/gitlab-ee!10515
parents 9621ace1 7dbf993a
......@@ -177,7 +177,7 @@ secondary domain, like changing Git remotes and API URLs.
```ruby
# Change the existing external_url configuration
external_url 'https://gitlab.example.com'
external_url 'https://<new_external_url>'
```
NOTE: **Note**
......@@ -242,11 +242,10 @@ and after that you also need two extra steps.
roles ['geo_primary_role']
##
# Primary and Secondary addresses
# - replace '198.51.100.1' with the public or VPC address of your Geo primary node
# - replace '198.51.100.2' with the public or VPC address of your Geo secondary node
# Allow PostgreSQL client authentication from the primary and secondary IPs. These IPs may be
# public or VPC addresses in CIDR format, for example ['198.51.100.1/32', '198.51.100.2/32']
##
postgresql['md5_auth_cidr_addresses'] = ['198.51.100.1/32', '198.51.100.2/32']
postgresql['md5_auth_cidr_addresses'] = ['<primary_node_ip>/32', '<secondary_node_ip>/32']
# Every secondary server needs to have its own slot so specify the number of secondary nodes you're going to have
postgresql['max_replication_slots'] = 1
......
......@@ -143,21 +143,19 @@ access to the **primary** node during the maintenance window.
all HTTP, HTTPS and SSH traffic to/from the **primary** node, **except** for your IP and
the **secondary** node's IP.
For instance, if your **secondary** node originates all its traffic from `5.6.7.8` and
your IP is `100.0.0.1`, you might run the following commands on the server(s)
making up your **primary** node:
For instance, you might run the following commands on the server(s) making up your **primary** node:
```sh
sudo iptables -A INPUT -p tcp -s 5.6.7.8 --destination-port 22 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 100.0.0.1 --destination-port 22 -j ACCEPT
sudo iptables -A INPUT -p tcp -s <secondary_node_ip> --destination-port 22 -j ACCEPT
sudo iptables -A INPUT -p tcp -s <your_ip> --destination-port 22 -j ACCEPT
sudo iptables -A INPUT --destination-port 22 -j REJECT
sudo iptables -A INPUT -p tcp -s 5.6.7.8 --destination-port 80 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 100.0.0.1 --destination-port 80 -j ACCEPT
sudo iptables -A INPUT -p tcp -s <secondary_node_ip> --destination-port 80 -j ACCEPT
sudo iptables -A INPUT -p tcp -s <your_ip> --destination-port 80 -j ACCEPT
sudo iptables -A INPUT --tcp-dport 80 -j REJECT
sudo iptables -A INPUT -p tcp -s 5.6.7.8 --destination-port 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 100.0.0.1 --destination-port 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -s <secondary_node_ip> --destination-port 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -s <your_ip> --destination-port 443 -j ACCEPT
sudo iptables -A INPUT --tcp-dport 443 -j REJECT
```
......
......@@ -109,8 +109,8 @@ keys must be manually replicated to the **secondary** node.
If you can access your **primary** node using the **root** user:
```sh
# Run this from the secondary node, change `primary-node-fqdn` for the IP or FQDN of the server
scp root@primary-node-fqdn:/etc/ssh/ssh_host_*_key* /etc/ssh
# Run this from the secondary node, change `<primary_node_fqdn>` for the IP or FQDN of the server
scp root@<primary_node_fqdn>:/etc/ssh/ssh_host_*_key* /etc/ssh
```
If you only have access through a user with **sudo** privileges:
......@@ -120,7 +120,7 @@ keys must be manually replicated to the **secondary** node.
sudo tar --transform 's/.*\///g' -zcvf ~/geo-host-key.tar.gz /etc/ssh/ssh_host_*_key*
# Run this from your secondary node:
scp user-with-sudo@primary-node-fqdn:geo-host-key.tar.gz .
scp <user_with_sudo>@<primary_node_fqdn>:geo-host-key.tar.gz .
tar zxvf ~/geo-host-key.tar.gz -C /etc/ssh
```
......
......@@ -138,12 +138,11 @@ You can safely skip this step if your **primary** node uses a CA-issued HTTPS ce
If your **primary** node is using a self-signed certificate for *HTTPS* support, you will
need to add that certificate to the **secondary** node's trust store. Retrieve the
certificate from the **primary** node and follow your distribution's instructions for
adding it to the **secondary** node's trust store. In Debian/Ubuntu, for example, with a
certificate file of `primary.geo.example.com.crt`, you would follow these steps:
adding it to the **secondary** node's trust store. In Debian/Ubuntu, you would follow these steps:
```sh
sudo -i
cp primary.geo.example.com.crt /usr/local/share/ca-certificates
cp <primary_node_certification_file> /usr/local/share/ca-certificates
update-ca-certificates
```
......
......@@ -45,10 +45,6 @@ The following guide assumes that:
replicating from), running Omnibus' PostgreSQL (or equivalent version), and
you have a new **secondary** server set up with the same versions of the OS,
PostgreSQL, and GitLab on all nodes.
- The IP of the **primary** server for our examples is `198.51.100.1`, whereas the
**secondary** node's IP is `198.51.100.2`. Note that the **primary** and **secondary** servers
**must** be able to communicate over these addresses. More on this in the
guide below.
CAUTION: **Warning:**
Geo works with streaming replication. Logical replication is not supported at this time.
......@@ -76,8 +72,8 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o
```sh
gitlab-ctl pg-password-md5 gitlab
# Enter password: mypassword
# Confirm password: mypassword
# Enter password: <your_password_here>
# Confirm password: <your_password_here>
# fca0b89a972d69f00eb3ec98a5838484
```
......@@ -85,12 +81,12 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o
```ruby
# Fill with the hash generated by `gitlab-ctl pg-password-md5 gitlab`
postgresql['sql_user_password'] = 'fca0b89a972d69f00eb3ec98a5838484'
postgresql['sql_user_password'] = '<md5_hash_of_your_password>'
# Every node that runs Unicorn or Sidekiq needs to have the database
# password specified as below. If you have a high-availability setup, this
# must be present in all application nodes.
gitlab_rails['db_password'] = 'mypassword'
gitlab_rails['db_password'] = '<your_password_here>'
```
1. Omnibus GitLab already has a [replication user]
......@@ -170,16 +166,15 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o
##
## Primary address
## - replace '198.51.100.1' with the public or VPC address of your Geo primary node
## - replace '<primary_node_ip>' with the public or VPC address of your Geo primary node
##
postgresql['listen_address'] = '198.51.100.1'
postgresql['listen_address'] = '<primary_node_ip>'
##
# Primary and Secondary addresses
# - replace '198.51.100.1' with the public or VPC address of your Geo primary node
# - replace '198.51.100.2' with the public or VPC address of your Geo secondary node
# Allow PostgreSQL client authentication from the primary and secondary IPs. These IPs may be
# public or VPC addresses in CIDR format, for example ['198.51.100.1/32', '198.51.100.2/32']
##
postgresql['md5_auth_cidr_addresses'] = ['198.51.100.1/32','198.51.100.2/32']
postgresql['md5_auth_cidr_addresses'] = ['<primary_node_ip>/32', '<secondary_node_ip>/32']
##
## Replication settings
......@@ -199,7 +194,7 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o
1. Optional: If you want to add another **secondary** node, the relevant setting would look like:
```ruby
postgresql['md5_auth_cidr_addresses'] = ['198.51.100.1/32', '198.51.100.2/32','198.51.100.3/32']
postgresql['md5_auth_cidr_addresses'] = ['<primary_node_ip>/32', '<secondary_node_ip>/32', '<another_secondary_node_ip>/32']
```
You may also want to edit the `wal_keep_segments` and `max_wal_senders` to
......@@ -274,7 +269,7 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o
1. [Check TCP connectivity][rake-maintenance] to the **primary** node's PostgreSQL server:
```sh
gitlab-rake gitlab:tcp_check[198.51.100.1,5432]
gitlab-rake gitlab:tcp_check[<primary_node_ip>,5432]
```
NOTE: **Note**:
......@@ -295,17 +290,29 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o
Install the `server.crt` file:
```sh
install -D -o gitlab-psql -g gitlab-psql -m 0400 -T server.crt ~gitlab-psql/.postgresql/root.crt
install \
-D \
-o gitlab-psql \
-g gitlab-psql \
-m 0400 \
-T server.crt ~gitlab-psql/.postgresql/root.crt
```
PostgreSQL will now only recognize that exact certificate when verifying TLS
connections. The certificate can only be replicated by someone with access
to the private key, which is **only** present on the **primary** node.
1. Test that the `gitlab-psql` user can connect to the **primary** node's database:
1. Test that the `gitlab-psql` user can connect to the **primary** node's database
(the default Omnibus database name is gitlabhq_production):
```sh
sudo -u gitlab-psql /opt/gitlab/embedded/bin/psql --list -U gitlab_replicator -d "dbname=gitlabhq_production sslmode=verify-ca" -W -h 198.51.100.1
sudo \
-u gitlab-psql /opt/gitlab/embedded/bin/psql \
--list \
-U gitlab_replicator \
-d "dbname=gitlabhq_production sslmode=verify-ca" \
-W \
-h <primary_node_ip>
```
When prompted enter the password you set in the first step for the
......@@ -333,17 +340,17 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o
##
## Secondary address
## - replace '198.51.100.2' with the public or VPC address of your Geo secondary node
## - replace '<secondary_node_ip>' with the public or VPC address of your Geo secondary node
##
postgresql['listen_address'] = '198.51.100.2'
postgresql['md5_auth_cidr_addresses'] = ['198.51.100.2/32']
postgresql['listen_address'] = '<secondary_node_ip>'
postgresql['md5_auth_cidr_addresses'] = ['<secondary_node_ip>/32']
##
## Database credentials password (defined previously in primary node)
## - replicate same values here as defined in primary node
##
postgresql['sql_user_password'] = 'fca0b89a972d69f00eb3ec98a5838484'
gitlab_rails['db_password'] = 'mypassword'
postgresql['sql_user_password'] = '<md5_hash_of_your_password>'
gitlab_rails['db_password'] = '<your_password_here>'
##
## Enable FDW support for the Geo Tracking Database (improves performance)
......@@ -399,7 +406,9 @@ data before running `pg_basebackup`.
Using the same slot name between two secondaries will break PostgreSQL replication.
```sh
gitlab-ctl replicate-geo-database --slot-name=secondary_example --host=198.51.100.1
gitlab-ctl replicate-geo-database \
--slot-name=<secondary_node_name> \
--host=<secondary_node_ip>
```
When prompted, enter the _plaintext_ password you set up for the `gitlab_replicator`
......@@ -443,10 +452,12 @@ work:
1. On the **primary** Geo database, enter the PostgreSQL on the console as an
admin user. If you are using an Omnibus-managed database, log onto the **primary**
node that is running the PostgreSQL database:
node that is running the PostgreSQL database (the default Omnibus database name is gitlabhq_production):
```sh
sudo -u gitlab-psql /opt/gitlab/embedded/bin/psql -h /var/opt/gitlab/postgresql gitlabhq_production
sudo \
-u gitlab-psql /opt/gitlab/embedded/bin/psql \
-h /var/opt/gitlab/postgresql gitlabhq_production
```
1. Then create the read-only user:
......
......@@ -69,13 +69,16 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o
CREATE USER gitlab_replicator;
--- Set/change a password and grants replication privilege
ALTER USER gitlab_replicator WITH REPLICATION ENCRYPTED PASSWORD 'replicationpasswordhere';
ALTER USER gitlab_replicator WITH REPLICATION ENCRYPTED PASSWORD '<replication_password>';
```
1. Make sure your the `gitlab` database user has a password defined:
```sh
sudo -u postgres psql -d template1 -c "ALTER USER gitlab WITH ENCRYPTED PASSWORD 'mydatabasepassword';"
sudo \
-u postgres psql \
-d template1 \
-c "ALTER USER gitlab WITH ENCRYPTED PASSWORD '<database_password>';"
```
1. Edit the content of `database.yml` in `production:` and add the password like the example below:
......@@ -90,7 +93,7 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o
database: gitlabhq_production
pool: 10
username: gitlab
password: mydatabasepassword
password: <database_password>
host: /var/opt/gitlab/geo-postgresql
```
......@@ -110,7 +113,14 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o
To generate a self-signed certificate and key, run this command:
```sh
openssl req -nodes -batch -x509 -newkey rsa:4096 -keyout server.key -out server.crt -days 3650
openssl req \
-nodes \
-batch \
-x509 \
-newkey rsa:4096 \
-keyout server.key \
-out server.crt \
-days 3650
```
This will create two files - `server.key` and `server.crt` - that you can
......@@ -137,7 +147,7 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o
(for Debian/Ubuntu that would be `/etc/postgresql/9.x/main/postgresql.conf`):
```
listen_address = '198.51.100.1'
listen_address = '<primary_node_ip>'
wal_level = hot_standby
max_wal_senders = 5
min_wal_size = 80MB
......@@ -171,19 +181,17 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o
`/etc/postgresql/9.x/main/pg_hba.conf`):
```sh
host all all 198.51.100.1/32 md5
host replication gitlab_replicator 198.51.100.2/32 md5
host all all <primary_node_ip>/32 md5
host replication gitlab_replicator <secondary_node_ip>/32 md5
```
Where `198.51.100.1` is the public IP address of the **primary** server, and `198.51.100.2`
the public IP address of the **secondary** one. If you want to add another
secondary, add one more row like the replication one and change the IP
address:
If you want to add another secondary, add one more row like the replication
one and change the IP address:
```sh
host all all 198.51.100.1/32 md5
host replication gitlab_replicator 198.51.100.2/32 md5
host replication gitlab_replicator 198.51.100.3/32 md5
host all all <primary_node_ip>/32 md5
host replication gitlab_replicator <secondary_node_ip>/32 md5
host replication gitlab_replicator <another_secondary_node_ip>/32 md5
```
1. Restart PostgreSQL for the changes to take effect.
......@@ -393,7 +401,7 @@ The replication process is now over.
```sql
-- NOTE: Use the password defined earlier
CREATE USER gitlab_geo_fdw WITH password 'mypassword';
CREATE USER gitlab_geo_fdw WITH password '<your_password_here>';
GRANT CONNECT ON DATABASE gitlabhq_production to gitlab_geo_fdw;
GRANT USAGE ON SCHEMA public TO gitlab_geo_fdw;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO gitlab_geo_fdw;
......
......@@ -116,10 +116,10 @@ To configure the connection to the external read-replica database and enable Log
# note this is shared between both databases,
# make sure you define the same password in both
gitlab_rails['db_password'] = 'mypassword'
gitlab_rails['db_password'] = '<your_password_here>'
gitlab_rails['db_username'] = 'gitlab'
gitlab_rails['db_host'] = 'my-database-read-replica.dbs.com'
gitlab_rails['db_host'] = '<database_read_replica_host>'
```
1. Save the file and [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure)
......@@ -156,10 +156,10 @@ the tracking database on port 5432.
```ruby
geo_secondary['db_username'] = 'gitlab_geo'
geo_secondary['db_password'] = 'my password'
geo_secondary['db_password'] = '<your_password_here>'
geo_secondary['db_host'] = '<change to the tracking DB host>'
geo_secondary['db_port'] = 5432 # change to the correct port
geo_secondary['db_host'] = '<tracking_database_host>'
geo_secondary['db_port'] = <tracking_database_port> # change to the correct port
geo_secondary['db_fdw'] = true # enable FDW
geo_postgresql['enable'] = false # don't use internal managed instance
```
......@@ -184,14 +184,14 @@ the tracking database on port 5432.
#!/bin/bash
# Secondary Database connection params:
DB_HOST="<change to the public IP or VPC private IP>"
DB_HOST="<public_ip_or_vpc_private_ip>"
DB_NAME="gitlabhq_production"
DB_USER="gitlab"
DB_PASS="my password"
DB_PASS="<your_password_here>"
DB_PORT="5432"
# Tracking Database connection params:
GEO_DB_HOST="<change to the public IP or VPC private IP>"
GEO_DB_HOST="<public_ip_or_vpc_private_ip>"
GEO_DB_NAME="gitlabhq_geo_production"
GEO_DB_USER="gitlab_geo"
GEO_DB_PORT="5432"
......
......@@ -160,22 +160,22 @@ following modifications:
## Configure the connection to the tracking DB. And disable application
## servers from running tracking databases.
##
geo_secondary['db_host'] = '10.1.4.1'
geo_secondary['db_password'] = 'plaintext Geo tracking DB password'
geo_secondary['db_host'] = '<geo_tracking_db_host>'
geo_secondary['db_password'] = '<geo_tracking_db_password>'
geo_postgresql['enable'] = false
##
## Configure connection to the streaming replica database, if you haven't
## already
##
gitlab_rails['db_host'] = '10.1.3.1'
gitlab_rails['db_password'] = 'plaintext DB password'
gitlab_rails['db_host'] = '<replica_database_host>'
gitlab_rails['db_password'] = '<replica_database_password>'
##
## Configure connection to Redis, if you haven't already
##
gitlab_rails['redis_host'] = '10.1.2.1'
gitlab_rails['redis_password'] = 'Redis password'
gitlab_rails['redis_host'] = '<redis_host>'
gitlab_rails['redis_password'] = '<redis_password>'
##
## If you are using custom users not managed by Omnibus, you need to specify
......
......@@ -140,7 +140,11 @@ Re-run `gitlab-ctl replicate-geo-database`, but include a larger value for
`--backup-timeout`:
```sh
sudo gitlab-ctl replicate-geo-database --host=primary.geo.example.com --slot-name=secondary_geo_example_com --backup-timeout=21600
sudo gitlab-ctl \
replicate-geo-database \
--host=<primary_node_hostname> \
--slot-name=<secondary_slot_name> \
--backup-timeout=21600
```
This will give the initial replication up to six hours to complete, rather than
......@@ -174,7 +178,7 @@ Slots where `active` is `f` are not active.
PostgreSQL console session:
```sql
SELECT pg_drop_replication_slot('name_of_extra_slot');
SELECT pg_drop_replication_slot('<name_of_extra_slot>');
```
## Very large repositories never successfully synchronize on the **secondary** node
......@@ -357,7 +361,7 @@ should see something like this:
following queries demonstrate how:
```sql
ALTER SERVER gitlab_secondary OPTIONS (SET host 'my-new-host');
ALTER SERVER gitlab_secondary OPTIONS (SET host '<my_new_host>');
ALTER SERVER gitlab_secondary OPTIONS (SET port 5432);
```
......@@ -383,13 +387,19 @@ should see something like this:
```sh
# Connect to the tracking database as the `gitlab_geo` user
sudo -u git /opt/gitlab/embedded/bin/psql -h /var/opt/gitlab/geo-postgresql -p 5431 -U gitlab_geo -W -d gitlabhq_geo_production
sudo \
-u git /opt/gitlab/embedded/bin/psql \
-h /var/opt/gitlab/geo-postgresql \
-p 5431 \
-U gitlab_geo \
-W \
-d gitlabhq_geo_production
```
If you need to correct the password, the following query shows how:
```sql
ALTER USER MAPPING FOR gitlab_geo SERVER gitlab_secondary OPTIONS (SET password 'my-new-password');
ALTER USER MAPPING FOR gitlab_geo SERVER gitlab_secondary OPTIONS (SET password '<my_new_password>');
```
If you change the user or password, you will also have to adjust the
......@@ -415,7 +425,13 @@ reload of the FDW schema. To manually reload the FDW schema:
the `gitlab_geo` user:
```sh
sudo -u git /opt/gitlab/embedded/bin/psql -h /var/opt/gitlab/geo-postgresql -p 5431 -U gitlab_geo -W -d gitlabhq_geo_production
sudo \
-u git /opt/gitlab/embedded/bin/psql \
-h /var/opt/gitlab/geo-postgresql \
-p 5431 \
-U gitlab_geo \
-W \
-d gitlabhq_geo_production
```
Be sure to adjust the port and hostname for your configuration. You
......
......@@ -34,8 +34,8 @@ authentication method.
```sh
gitlab-ctl pg-password-md5 gitlab
# Enter password: mypassword
# Confirm password: mypassword
# Enter password: <your_password_here>
# Confirm password: <your_password_here>
# fca0b89a972d69f00eb3ec98a5838484
```
......@@ -43,12 +43,12 @@ authentication method.
```ruby
# Fill with the hash generated by `gitlab-ctl pg-password-md5 gitlab`
postgresql['sql_user_password'] = 'fca0b89a972d69f00eb3ec98a5838484'
postgresql['sql_user_password'] = '<md5_hash_of_your_password>'
# Every node that runs Unicorn or Sidekiq needs to have the database
# password specified as below. If you have a high-availability setup, this
# must be present in all application nodes.
gitlab_rails['db_password'] = 'mypassword'
gitlab_rails['db_password'] = '<your_password_here>'
```
Still in the configuration file, locate and remove the `trust_auth_cidr_address`:
......@@ -68,19 +68,18 @@ authentication method.
```ruby
# Fill with the hash generated by `gitlab-ctl pg-password-md5 gitlab`
postgresql['sql_user_password'] = 'fca0b89a972d69f00eb3ec98a5838484'
postgresql['sql_user_password'] = '<md5_hash_of_your_password>'
# Every node that runs Unicorn or Sidekiq needs to have the database
# password specified as below. If you have a high-availability setup, this
# must be present in all application nodes.
gitlab_rails['db_password'] = 'mypassword'
gitlab_rails['db_password'] = '<your_password_here>'
# Enable Foreign Data Wrapper
geo_secondary['db_fdw'] = true
# Secondary address
# - replace '5.6.7.8' with the secondary public or VPC address
postgresql['md5_auth_cidr_addresses'] = ['5.6.7.8/32']
# Secondary address in CIDR format, for example '5.6.7.8/32'
postgresql['md5_auth_cidr_addresses'] = ['<secondary_node_ip>/32']
```
Still in the configuration file, locate and remove the `trust_auth_cidr_address`:
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment