<!-- Un-comment the line for the applicable doc issue type to add its label.
Note that all text on that line is deleted upon issue creation. -->
<!-- /label ~"docs:fix" - Correction or clarification needed. -->
<!-- /label ~"docs:new" - New doc needed to cover a new topic or use case. -->
<!-- /label ~"docs:improvement" - Improving an existing doc; e.g. adding a diagram, adding or rewording text, resolving redundancies, cross-linking, etc. -->
<!-- /label ~"docs:revamp" - Review a page or group of pages in order to plan and implement major improvements/rewrites. -->
@@ -37,4 +37,4 @@ All reviewers can help ensure accuracy, clarity, completeness, and adherence to
1. [ ] Ensure a release milestone is set and that you merge the equivalent EE MR before the CE MR if both exist.
1.[ ] If there has not been a technical writer review, [create an issue for one using the Doc Review template](https://gitlab.com/gitlab-org/gitlab-ce/issues/new?issuable_template=Doc%20Review).
- [Documentation workflows](https://docs.gitlab.com/ee/development/documentation/workflow.html) for information on when to assign a merge request for review.
MARKDOWN
unlessgitlab.mr_labels.include?('Documentation')
warn'This merge request is missing the ~Documentation label.'
unlessgitlab.mr_labels.include?('documentation')
warn'This merge request is missing the ~documentation label.'
@@ -43,9 +43,9 @@ attachments / avatars and the whole database. This means user accounts,
issues, merge requests, groups, project data, etc., will be available for
query.
## Can I git push to a **secondary** node?
## Can I `git push` to a **secondary** node?
Yes! Pushing directly to a **secondary** node (for both HTTP and SSH, including git-lfs) was [introduced](https://about.gitlab.com/2018/09/22/gitlab-11-3-released/) in [GitLab Premium](https://about.gitlab.com/pricing/#self-managed) 11.3.
Yes! Pushing directly to a **secondary** node (for both HTTP and SSH, including Git LFS) was [introduced](https://about.gitlab.com/2018/09/22/gitlab-11-3-released/) in [GitLab Premium](https://about.gitlab.com/pricing/#self-managed) 11.3.
## How long does it take to have a commit replicated to a **secondary** node?
- Replicate repositories, LFS Objects, and Attachments (HTTPS + JWT).
- Since GitLab Premium 10.0, the **primary** node no longer talks to **secondary** nodes to notify for changes (API).
- Pushing directly to a **secondary** node (for both HTTP and SSH, including git-lfs) was [introduced](https://about.gitlab.com/2018/09/22/gitlab-11-3-released/) in [GitLab Premium](https://about.gitlab.com/pricing/#self-managed) 11.3.
- Pushing directly to a **secondary** node (for both HTTP and SSH, including Git LFS) was [introduced](https://about.gitlab.com/2018/09/22/gitlab-11-3-released/) in [GitLab Premium](https://about.gitlab.com/pricing/#self-managed) 11.3.
- There are [limitations](#current-limitations) in the current implementation.
### Architecture
...
...
@@ -240,7 +240,7 @@ This list of limitations only reflects the latest version of GitLab. If you are
- Pushing directly to a **secondary** node redirects (for HTTP) or proxies (for SSH) the request to the **primary** node instead of [handling it directly](https://gitlab.com/gitlab-org/gitlab/issues/1381), except when using Git over HTTP with credentials embedded within the URI. For example, `https://user:password@secondary.tld`.
- The **primary** node has to be online for OAuth login to happen. Existing sessions and Git are not affected.
- The installation takes multiple manual steps that together can take about an hour depending on circumstances. We are working on improving this experience. See [gitlab-org/omnibus-gitlab#2978](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2978) for details.
- The installation takes multiple manual steps that together can take about an hour depending on circumstances. We are working on improving this experience. See [Omnibus GitLab issue #2978](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2978) for details.
- Real-time updates of issues/merge requests (for example, via long polling) doesn't work on the **secondary** node.
- [Selective synchronization](configuration.md#selective-synchronization) applies only to files and repositories. Other datasets are replicated to the **secondary** node in full, making it inappropriate for use as an access control mechanism.
- Object pools for forked project deduplication work only on the **primary** node, and are duplicated on the **secondary** node.
After you set up the [database replication and configure the Geo nodes][req], use your closest GitLab node as you would a normal standalone GitLab instance.
Pushing directly to a **secondary** node (for both HTTP, SSH including git-lfs) was [introduced](https://about.gitlab.com/2018/09/22/gitlab-11-3-released/) in [GitLab Premium](https://about.gitlab.com/pricing/#self-managed) 11.3.
Pushing directly to a **secondary** node (for both HTTP, SSH including Git LFS) was [introduced](https://about.gitlab.com/2018/09/22/gitlab-11-3-released/) in [GitLab Premium](https://about.gitlab.com/pricing/#self-managed) 11.3.
Example of the output you will see when pushing to a **secondary** node:
- Consul - Database service discovery and health checks/failover
- PGBouncer - Database pool manager
- PgBouncer - Database pool manager
- Redis - Key/Value store (User sessions, cache, queue for Sidekiq)
- Sentinel - Redis health check/failover manager
- Gitaly - Provides high-level RPC access to Git repositories
...
...
@@ -138,7 +138,7 @@ the contention.
- 3 PostgreSQL nodes
- 2 Redis nodes
- 3 Consul/Sentinel nodes
- 2 or more GitLab application nodes (Unicorn, Workhorse, Sidekiq, PGBouncer)
- 2 or more GitLab application nodes (Unicorn, Workhorse, Sidekiq, PgBouncer)
- 1 NFS/Gitaly server
- 1 Monitoring node (Prometheus, Grafana)
...
...
@@ -165,7 +165,7 @@ contention due to certain workloads.
#### Reference Architecture
-**Supported Users (approximate):** 10,000
-**Known Issues:** While validating the reference architecture, slow endpoints were discovered and are being investigated. [gitlab-org/gitlab-foss/issues/64335](https://gitlab.com/gitlab-org/gitlab-foss/issues/64335)
-**Known Issues:** While validating the reference architecture, slow endpoints were discovered and are being investigated. [See issue #64335](https://gitlab.com/gitlab-org/gitlab-foss/issues/64335)
The Support and Quality teams built, performance tested, and validated an
environment that supports about 10,000 users. The specifications below are a
As part of its High Availability stack, GitLab Premium includes a bundled version of [Consul](https://www.consul.io/) that can be managed through `/etc/gitlab/gitlab.rb`.
A Consul cluster consists of multiple server agents, as well as client agents that run on other nodes which need to talk to the consul cluster.
A Consul cluster consists of multiple server agents, as well as client agents that run on other nodes which need to talk to the Consul cluster.
## Prerequisites
...
...
@@ -96,7 +96,7 @@ Ideally all nodes will have a `Status` of `alive`.
**Note**: This section only applies to server agents. It is safe to restart client agents whenever needed.
If it is necessary to restart the server cluster, it is important to do this in a controlled fashion in order to maintain quorum. If quorum is lost, you will need to follow the consul [outage recovery](#outage-recovery) process to recover the cluster.
If it is necessary to restart the server cluster, it is important to do this in a controlled fashion in order to maintain quorum. If quorum is lost, you will need to follow the Consul [outage recovery](#outage-recovery) process to recover the cluster.
To be safe, we recommend you only restart one server agent at a time to ensure the cluster remains intact.
...
...
@@ -129,7 +129,7 @@ To fix this:
1. Run `gitlab-ctl reconfigure`
If you still see the errors, you may have to [erase the consul database and reinitialize](#recreate-from-scratch) on the affected node.
If you still see the errors, you may have to [erase the Consul database and reinitialize](#recreate-from-scratch) on the affected node.
### Consul agents do not start - Multiple private IPs
...
...
@@ -162,7 +162,7 @@ If you lost enough server agents in the cluster to break quorum, then the cluste
#### Recreate from scratch
By default, GitLab does not store anything in the consul cluster that cannot be recreated. To erase the consul database and reinitialize
By default, GitLab does not store anything in the Consul cluster that cannot be recreated. To erase the Consul database and reinitialize
```
# gitlab-ctl stop consul
...
...
@@ -174,4 +174,4 @@ After this, the cluster should start back up, and the server agents rejoin. Shor
#### Recover a failed cluster
If you have taken advantage of consul to store other data, and want to restore the failed cluster, please follow the [Consul guide](https://www.consul.io/docs/guides/outage.html) to recover a failed cluster.
If you have taken advantage of Consul to store other data, and want to restore the failed cluster, please follow the [Consul guide](https://www.consul.io/docs/guides/outage.html) to recover a failed cluster.
@@ -109,7 +109,7 @@ For more details on another person's experience with EFS, see
## Avoid using CephFS and GlusterFS
GitLab strongly recommends against using CephFS and GlusterFS.
These distributed file systems are not well-suited for GitLab's input/output access patterns because git uses many small files and access times and file locking times to propagate will make git activity very slow.
These distributed file systems are not well-suited for GitLab's input/output access patterns because Git uses many small files and access times and file locking times to propagate will make Git activity very slow.
## Avoid using PostgreSQL with NFS
...
...
@@ -147,7 +147,7 @@ Note there are several options that you should consider using:
## A single NFS mount
It's recommended to nest all gitlab data dirs within a mount, that allows automatic
It's recommended to nest all GitLab data dirs within a mount, that allows automatic
restore of backups without manually moving existing data.
As part of its High Availability stack, GitLab Premium includes a bundled version of [Pgbouncer](https://pgbouncer.github.io/) that can be managed through `/etc/gitlab/gitlab.rb`.
As part of its High Availability stack, GitLab Premium includes a bundled version of [PgBouncer](https://pgbouncer.github.io/) that can be managed through `/etc/gitlab/gitlab.rb`.
In a High Availability setup, Pgbouncer is used to seamlessly migrate database connections between servers in a failover scenario.
In a High Availability setup, PgBouncer is used to seamlessly migrate database connections between servers in a failover scenario.
Additionally, it can be used in a non-HA setup to pool connections, speeding up response time while reducing resource usage.
It is recommended to run pgbouncer alongside the `gitlab-rails` service, or on its own dedicated node in a cluster.
It is recommended to run PgBouncer alongside the `gitlab-rails` service, or on its own dedicated node in a cluster.
## Operations
### Running Pgbouncer as part of an HA GitLab installation
### Running PgBouncer as part of an HA GitLab installation
1. Make sure you collect [`CONSUL_SERVER_NODES`](database.md#consul-information), [`CONSUL_PASSWORD_HASH`](database.md#consul-information), and [`PGBOUNCER_PASSWORD_HASH`](database.md#pgbouncer-information) before executing the next step.
1. Edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section:
```ruby
# Disable all components except Pgbouncer and Consul agent
# Disable all components except PgBouncer and Consul agent
1. Ensure the node is talking to the current master:
...
...
@@ -100,7 +100,7 @@ It is recommended to run pgbouncer alongside the `gitlab-rails` service, or on i
(2 rows)
```
### Running Pgbouncer as part of a non-HA GitLab installation
### Running PgBouncer as part of a non-HA GitLab installation
1. Generate PGBOUNCER_USER_PASSWORD_HASH with the command `gitlab-ctl pg-password-md5 pgbouncer`
...
...
@@ -119,7 +119,7 @@ It is recommended to run pgbouncer alongside the `gitlab-rails` service, or on i
**Note:** If the database was already running, it will need to be restarted after reconfigure by running `gitlab-ctl restart postgresql`.
1. On the node you are running pgbouncer on, make sure the following is set in `/etc/gitlab/gitlab.rb`
1. On the node you are running PgBouncer on, make sure the following is set in `/etc/gitlab/gitlab.rb`
```ruby
pgbouncer['enable']=true
...
...
@@ -134,7 +134,7 @@ It is recommended to run pgbouncer alongside the `gitlab-rails` service, or on i
1. Run `gitlab-ctl reconfigure`
1. On the node running unicorn, make sure the following is set in `/etc/gitlab/gitlab.rb`
1. On the node running Unicorn, make sure the following is set in `/etc/gitlab/gitlab.rb`
```ruby
gitlab_rails['db_host']='PGBOUNCER_HOST'
...
...
@@ -144,13 +144,13 @@ It is recommended to run pgbouncer alongside the `gitlab-rails` service, or on i
1. Run `gitlab-ctl reconfigure`
1. At this point, your instance should connect to the database through pgbouncer. If you are having issues, see the [Troubleshooting](#troubleshooting) section
1. At this point, your instance should connect to the database through PgBouncer. If you are having issues, see the [Troubleshooting](#troubleshooting) section
## Enable Monitoring
> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3786) in GitLab 12.0.
If you enable Monitoring, it must be enabled on **all**pgbouncer servers.
If you enable Monitoring, it must be enabled on **all**PgBouncer servers.
1. Create/edit `/etc/gitlab/gitlab.rb` and add the following configuration:
...
...
@@ -173,11 +173,11 @@ If you enable Monitoring, it must be enabled on **all** pgbouncer servers.
1. Run `sudo gitlab-ctl reconfigure` to compile the configuration.
### Interacting with pgbouncer
### Interacting with PgBouncer
#### Administrative console
As part of omnibus-gitlab, we provide a command `gitlab-ctl pgb-console` to automatically connect to the pgbouncer administrative console. Please see the [pgbouncer documentation](https://pgbouncer.github.io/usage.html#admin-console) for detailed instructions on how to interact with the console.
As part of Omnibus GitLab, we provide a command `gitlab-ctl pgb-console` to automatically connect to the PgBouncer administrative console. Please see the [PgBouncer documentation](https://pgbouncer.github.io/usage.html#admin-console) for detailed instructions on how to interact with the console.
To start a session, run
...
...
@@ -235,7 +235,7 @@ ote_pid | tls
## Troubleshooting
In case you are experiencing any issues connecting through pgbouncer, the first place to check is always the logs:
In case you are experiencing any issues connecting through PgBouncer, the first place to check is always the logs:
@@ -77,7 +77,7 @@ and default merge request template will assist you with following this process.
For issues requiring any new or updated documentation, the Product Manager (PM)
must:
- Add the `Documentation` label.
- Add the ~documentation label.
- Confirm or add the [documentation requirements](#documentation-requirements-in-feature-issues).
- Ensure the issue contains any new or updated feature name, overview/description,
and use cases, as required per the [documentation structure and template](structure.md), when applicable.
...
...
@@ -92,7 +92,7 @@ do the following:
#### Authoring
As a developer, if a ~feature issue also contains the ~Documentation label, you must ship the new or updated documentation with the code of the feature. The documentation is an essential part of the product.
As a developer, if a ~feature issue also contains the ~documentation label, you must ship the new or updated documentation with the code of the feature. The documentation is an essential part of the product.
Technical writers are happy to help, as requested and planned on an issue-by-issue basis.
For feature issues requiring documentation, follow the process below unless otherwise agreed with the product manager and technical writer for a given issue:
@@ -16,7 +16,7 @@ All Geo nodes have the following settings:
| Setting | Description |
| --------| ----------- |
| Primary | This marks a Geo Node as **primary** node. There can be only one **primary** node; make sure that you first add the **primary** node and then all the others. |
| Name | The unique identifier for the Geo node. Must match the setting `gitlab_rails[geo_node_name]` in `/etc/gitlab/gitlab.rb`. The setting defaults to `external_url` with a trailing slash. |
| Name | The unique identifier for the Geo node. Must match the setting `gitlab_rails['geo_node_name']` in `/etc/gitlab/gitlab.rb`. The setting defaults to `external_url` with a trailing slash. |
| URL | The instance's user-facing URL. |
The node you're reading from is indicated with a green `Current node` label, and
...
...
@@ -71,7 +71,7 @@ terminated at the load balancer.
In GitLab 11.11, **secondary** nodes can use identical external URLs as long as
a unique `name` is set for each Geo node. The `gitlab.rb` setting
`gitlab_rails[geo_node_name]` must:
`gitlab_rails['geo_node_name']` must:
- Be set for each GitLab instance that runs `unicorn`, `sidekiq`, or `geo_logcursor`.