Commit aa0dd496 authored by Mike Lewis's avatar Mike Lewis

Merge branch 'gy-ha-docs-cleanup' into 'master'

Cleanup Scaling / HA docs page

Closes #205634, #205632, and #205635

See merge request gitlab-org/gitlab!25422
parents fecff5e3 9ae21b39
...@@ -6,9 +6,9 @@ components can read or write Git data. GitLab components that access Git ...@@ -6,9 +6,9 @@ components can read or write Git data. GitLab components that access Git
repositories (GitLab Rails, GitLab Shell, GitLab Workhorse, etc.) act as clients repositories (GitLab Rails, GitLab Shell, GitLab Workhorse, etc.) act as clients
to Gitaly. End users do not have direct access to Gitaly. to Gitaly. End users do not have direct access to Gitaly.
In the rest of this page, Gitaly server is referred to the standalone node that On this page, *Gitaly server* refers to a standalone node that only runs Gitaly
only runs Gitaly, and Gitaly client to the GitLab Rails node that runs all other and *Gitaly client* is a GitLab Rails app node that runs all other processes
processes except Gitaly. except Gitaly.
## Architecture ## Architecture
...@@ -20,7 +20,7 @@ Here's a high-level architecture overview of how Gitaly is used. ...@@ -20,7 +20,7 @@ Here's a high-level architecture overview of how Gitaly is used.
The Gitaly service itself is configured via a [TOML configuration file](reference.md). The Gitaly service itself is configured via a [TOML configuration file](reference.md).
In case you want to change some of its settings: If you want to change any of its settings:
**For Omnibus GitLab** **For Omnibus GitLab**
...@@ -54,10 +54,6 @@ scenario, the [new repository indexer](../../integration/elasticsearch.md#elasti ...@@ -54,10 +54,6 @@ scenario, the [new repository indexer](../../integration/elasticsearch.md#elasti
needs to be enabled in your GitLab configuration. [Since GitLab v12.3](https://gitlab.com/gitlab-org/gitlab/issues/6481), needs to be enabled in your GitLab configuration. [Since GitLab v12.3](https://gitlab.com/gitlab-org/gitlab/issues/6481),
the new indexer becomes the default and no configuration is required. the new indexer becomes the default and no configuration is required.
NOTE: **Note:** While Gitaly can be used as a replacement for NFS, it's not recommended
to use EFS as it may impact GitLab's performance. Review the [relevant documentation](../high_availability/nfs.md#avoid-using-awss-elastic-file-system-efs)
for more details.
### Network architecture ### Network architecture
The following list depicts what the network architecture of Gitaly is: The following list depicts what the network architecture of Gitaly is:
...@@ -568,30 +564,6 @@ server with the following settings. ...@@ -568,30 +564,6 @@ server with the following settings.
1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source). 1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source).
## Eliminating NFS altogether
If you are planning to use Gitaly without NFS for your storage needs
and want to eliminate NFS from your environment altogether, there are
a few things that you need to do:
1. Make sure the [`git` user home directory](https://docs.gitlab.com/omnibus/settings/configuration.html#moving-the-home-directory-for-a-user) is on local disk.
1. Configure [database lookup of SSH keys](../operations/fast_ssh_key_lookup.md)
to eliminate the need for a shared `authorized_keys` file.
1. Configure [object storage for job artifacts](../job_artifacts.md#using-object-storage)
including [incremental logging](../job_logs.md#new-incremental-logging-architecture).
1. Configure [object storage for LFS objects](../lfs/lfs_administration.md#storing-lfs-objects-in-remote-object-storage).
1. Configure [object storage for uploads](../uploads.md#using-object-storage-core-only).
1. Configure [object storage for merge request diffs](../merge_request_diffs.md#using-object-storage).
1. Configure [object storage for packages](../packages/index.md#using-object-storage) (optional feature).
1. Configure [object storage for dependency proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature).
1. Configure [object storage for Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage) (optional feature).
NOTE: **Note:**
One current feature of GitLab that still requires a shared directory (NFS) is
[GitLab Pages](../../user/project/pages/index.md).
There is [work in progress](https://gitlab.com/gitlab-org/gitlab-pages/issues/196)
to eliminate the need for NFS to support GitLab Pages.
## Limiting RPC concurrency ## Limiting RPC concurrency
It can happen that CI clone traffic puts a large strain on your Gitaly It can happen that CI clone traffic puts a large strain on your Gitaly
......
...@@ -22,11 +22,9 @@ If you use a cloud-managed service, or provide your own PostgreSQL: ...@@ -22,11 +22,9 @@ If you use a cloud-managed service, or provide your own PostgreSQL:
1. Configure the GitLab application servers with the appropriate details. 1. Configure the GitLab application servers with the appropriate details.
This step is covered in [Configuring GitLab for HA](gitlab.md). This step is covered in [Configuring GitLab for HA](gitlab.md).
## PostgreSQL in a Scaled Environment ## PostgreSQL in a Scaled and Highly Available Environment
This section is relevant for [Scaled Architecture](README.md#scalable-architecture-examples) This section is relevant for [Scalable and Highly Available Setups](README.md).
environments including [Basic Scaling](README.md#basic-scaling) and
[Full Scaling](README.md#full-scaling).
### Provide your own PostgreSQL instance **(CORE ONLY)** ### Provide your own PostgreSQL instance **(CORE ONLY)**
...@@ -94,23 +92,6 @@ deploy the bundled PostgreSQL. ...@@ -94,23 +92,6 @@ deploy the bundled PostgreSQL.
Advanced configuration options are supported and can be added if Advanced configuration options are supported and can be added if
needed. needed.
Continue configuration of other components by going
[back to Scaled Architectures](README.md#scalable-architecture-examples)
## PostgreSQL with High Availability
This section is relevant for [High Availability Architecture](README.md#high-availability-architecture-examples)
environments including [Horizontal](README.md#horizontal),
[Hybrid](README.md#hybrid), and
[Fully Distributed](README.md#fully-distributed).
### Provide your own PostgreSQL instance **(CORE ONLY)**
If you want to use your own deployed PostgreSQL instance(s),
see [Provide your own PostgreSQL instance](#provide-your-own-postgresql-instance-core-only)
for more details. However, you can use the GitLab Omnibus package to easily
deploy the bundled PostgreSQL.
### High Availability with GitLab Omnibus **(PREMIUM ONLY)** ### High Availability with GitLab Omnibus **(PREMIUM ONLY)**
> Important notes: > Important notes:
......
...@@ -11,18 +11,15 @@ should consider using Gitaly on a separate node. ...@@ -11,18 +11,15 @@ should consider using Gitaly on a separate node.
See the [Gitaly HA Epic](https://gitlab.com/groups/gitlab-org/-/epics/289) to See the [Gitaly HA Epic](https://gitlab.com/groups/gitlab-org/-/epics/289) to
track plans and progress toward high availability support. track plans and progress toward high availability support.
This document is relevant for [Scaled Architecture](README.md#scalable-architecture-examples) This document is relevant for [Scalable and Highly Available Setups](README.md).
environments and [High Availability Architecture](README.md#high-availability-architecture-examples).
## Running Gitaly on its own server ## Running Gitaly on its own server
See [Running Gitaly on its own server](../gitaly/index.md#running-gitaly-on-its-own-server) See [Running Gitaly on its own server](../gitaly/index.md#running-gitaly-on-its-own-server)
in Gitaly documentation. in Gitaly documentation.
Continue configuration of other components by going back to: Continue configuration of other components by going back to the
[Scaling and High Availability](README.md#gitlab-components-and-configuration-instructions) page.
- [Scaled Architectures](README.md#scalable-architecture-examples)
- [High Availability Architectures](README.md#high-availability-architecture-examples)
## Enable Monitoring ## Enable Monitoring
......
---
type: reference
---
# Cloud Object Storage
GitLab supports utilizing a Cloud Object Storage service over [NFS](nfs.md) for holding
numerous types of data. This is recommended in larger setups as object storage is
typically much more performant and reliable.
For configuring GitLab to use Object Storage refer to the following guides:
1. Make sure the [`git` user home directory](https://docs.gitlab.com/omnibus/settings/configuration.html#moving-the-home-directory-for-a-user) is on local disk.
1. Configure [database lookup of SSH keys](../operations/fast_ssh_key_lookup.md)
to eliminate the need for a shared `authorized_keys` file.
1. Configure [object storage for job artifacts](../job_artifacts.md#using-object-storage)
including [incremental logging](../job_logs.md#new-incremental-logging-architecture).
1. Configure [object storage for LFS objects](../lfs/lfs_administration.md#storing-lfs-objects-in-remote-object-storage).
1. Configure [object storage for uploads](../uploads.md#using-object-storage-core-only).
1. Configure [object storage for merge request diffs](../merge_request_diffs.md#using-object-storage).
1. Configure [object storage for packages](../packages/index.md#using-object-storage) (optional feature).
1. Configure [object storage for dependency proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature).
NOTE: **Note:**
One current feature of GitLab that still requires a shared directory (NFS) is
[GitLab Pages](../../user/project/pages/index.md).
There is [work in progress](https://gitlab.com/gitlab-org/gitlab-pages/issues/196)
to eliminate the need for NFS to support GitLab Pages.
...@@ -20,11 +20,9 @@ The following are the requirements for providing your own Redis instance: ...@@ -20,11 +20,9 @@ The following are the requirements for providing your own Redis instance:
Note the Redis node's IP address or hostname, port, and password (if required). Note the Redis node's IP address or hostname, port, and password (if required).
These will be necessary when configuring the GitLab application servers later. These will be necessary when configuring the GitLab application servers later.
## Redis in a Scaled Environment ## Redis in a Scaled and Highly Available Environment
This section is relevant for [Scaled Architecture](README.md#scalable-architecture-examples) This section is relevant for [Scalable and Highly Available Setups](README.md).
environments including [Basic Scaling](README.md#basic-scaling) and
[Full Scaling](README.md#full-scaling).
### Provide your own Redis instance **(CORE ONLY)** ### Provide your own Redis instance **(CORE ONLY)**
...@@ -85,22 +83,8 @@ Omnibus: ...@@ -85,22 +83,8 @@ Omnibus:
Advanced configuration options are supported and can be added if Advanced configuration options are supported and can be added if
needed. needed.
Continue configuration of other components by going Continue configuration of other components by going back to the
[back to Scaled Architectures](README.md#scalable-architecture-examples) [Scaling and High Availability](README.md#gitlab-components-and-configuration-instructions) page.
## Redis with High Availability
This section is relevant for [High Availability Architecture](README.md#high-availability-architecture-examples)
environments including [Horizontal](README.md#horizontal),
[Hybrid](README.md#hybrid), and
[Fully Distributed](README.md#fully-distributed).
### Provide your own Redis instance **(CORE ONLY)**
If you want to use your own deployed Redis instance(s),
see [Provide your own Redis instance](#provide-your-own-redis-instance-core-only)
for more details. However, you can use the GitLab Omnibus package to easily
deploy the bundled Redis.
### High Availability with GitLab Omnibus **(PREMIUM ONLY)** ### High Availability with GitLab Omnibus **(PREMIUM ONLY)**
......
...@@ -260,7 +260,7 @@ If the database size is less than 500 MiB, and the size of all hosted repos is l ...@@ -260,7 +260,7 @@ If the database size is less than 500 MiB, and the size of all hosted repos is l
CAUTION: **Warning**: CAUTION: **Warning**:
Performing asynchronous indexing will generate a lot of Sidekiq jobs. Performing asynchronous indexing will generate a lot of Sidekiq jobs.
Make sure to prepare for this task by either [Horizontally Scaling](../administration/high_availability/README.md#basic-scaling) Make sure to prepare for this task by having a [Scalable and Highly Available Setup](README.md)
or creating [extra Sidekiq processes](../administration/operations/extra_sidekiq_processes.md) or creating [extra Sidekiq processes](../administration/operations/extra_sidekiq_processes.md)
1. [Configure your Elasticsearch host and port](#enabling-elasticsearch). 1. [Configure your Elasticsearch host and port](#enabling-elasticsearch).
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment