Skip to content

G
gitlab-ce
Commit 95f43e44 authored by Evan Read's avatar Evan Read
Browse files
Options

Merge branch 'smh-document-metadata' into 'master' 

Document 'praefect metadata' subcommand

See merge request gitlab-org/gitlab!75260
parents f419a07e 7fd4aae6
Hide whitespace changes
Inline Side-by-side
Showing with 101 additions and 11 deletions
doc/administration/gitaly/praefect.md
View file @ 95f43e44
...@@ -1332,6 +1332,90 @@ To enable writes again in GitLab 13.0 to 14.0, an administrator can: ...@@ -1332,6 +1332,90 @@ To enable writes again in GitLab 13.0 to 14.0, an administrator can:
[accept data loss](#enable-writes-or-accept-data-loss) if necessary, depending on the version of [accept data loss](#enable-writes-or-accept-data-loss) if necessary, depending on the version of
GitLab. GitLab.
## Retrieve repository metadata
> [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/3481) in GitLab 14.6.
Gitaly Cluster maintains a [metadata database](index.md#components) about the repositories stored on the cluster. Use the `praefect metadata` subcommand
to inspect the metadata for troubleshooting.
You can retrieve a repository's metadata by its Praefect-assigned repository ID:
```shell
sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml metadata -repository-id <repository-id>
```
You can also retrieve a repository's metadata by its virtual storage and relative path:
```shell
sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml metadata -virtual-storage <virtual-storage> -relative-path <relative-path>
```
### Examples
To retrieve the metadata for a repository with a Praefect-assigned repository ID of 1:
```shell
sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml metadata -repository-id 1
```
To retrieve the metadata for a repository with virtual storage `default` and relative path `@hashed/b1/7e/b17ef6d19c7a5b1ee83b907c595526dcb1eb06db8227d650d5dda0a9f4ce8cd9.git`:
```shell
sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml metadata -virtual-storage default -relative-path @hashed/b1/7e/b17ef6d19c7a5b1ee83b907c595526dcb1eb06db8227d650d5dda0a9f4ce8cd9.git
```
Either of these examples retrieve the following metadata for an example repository:
```plaintext
Repository ID: 54771
Virtual Storage: "default"
Relative Path: "@hashed/b1/7e/b17ef6d19c7a5b1ee83b907c595526dcb1eb06db8227d650d5dda0a9f4ce8cd9.git"
Replica Path: "@hashed/b1/7e/b17ef6d19c7a5b1ee83b907c595526dcb1eb06db8227d650d5dda0a9f4ce8cd9.git"
Primary: "gitaly-1"
Generation: 1
Replicas:
- Storage: "gitaly-1"
Assigned: true
Generation: 1, fully up to date
Healthy: true
Valid Primary: true
- Storage: "gitaly-2"
Assigned: true
Generation: 0, behind by 1 changes
Healthy: true
Valid Primary: false
- Storage: "gitaly-3"
Assigned: true
Generation: replica not yet created
Healthy: false
Valid Primary: false
```
### Available metadata
The metadata retrieved by `praefect metadata` includes the fields in the following tables.
| Field | Description |
|:------------------|:-------------------------------------------------------------------------------------------------------------------|
| `Repository ID` | Permanent unique ID assigned to the repository by Praefect. Different to the ID GitLab uses for repositories. |
| `Virtual Storage` | Name of the virtual storage the repository is stored in. |
| `Relative Path` | Repository's path in the virtual storage. |
| `Replica Path` | Where on the Gitaly node's disk the repository's replicas are stored. |
| `Primary` | Current primary of the repository. |
| `Generation` | Used by Praefect to track repository changes. Each write in the repository increments the repository's generation. |
| `Replicas` | A list of replicas that exist or are expected to exist. |
For each replica, the following metadata is available:
| `Replicas` Field | Description |
|:-----------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `Storage` | Name of the Gitaly storage that contains the replica. |
| `Assigned` | Indicates whether the replica is expected to exist in the storage. Can be `false` if a Gitaly node is removed from the cluster or if the storage contains an extra copy after the repository's replication factor was decreased. |
| `Generation` | Latest confirmed generation of the replica. It indicates:<br><br>- The replica is fully up to date if the generation matches the repository's generation.<br>- The replica is outdated if the replica's generation is less than the repository's generation.<br>- `replica not yet created` if the replica does not yet exist at all on the storage. |
| `Healthy` | Indicates whether the Gitaly node that is hosting this replica is considered healthy by the consensus of Praefect nodes. |
| `Valid Primary` | Indicates whether the replica is fit to serve as the primary node. If the repository's primary is not a valid primary, a failover occurs on the next write to the repository if there is another replica that is a valid primary. A replica is a valid primary if:<br><br>- It is stored on a healthy Gitaly node.<br>- It is fully up to date.<br>- It is not targeted by a pending deletion job from decreasing replication factor.<br>- It is assigned. |
## Unavailable repositories ## Unavailable repositories
> - From GitLab 13.0 through 14.0, repositories became read-only if they were outdated on the primary but fully up to date on a healthy secondary. `dataloss` sub-command displays read-only repositories by default through these versions. > - From GitLab 13.0 through 14.0, repositories became read-only if they were outdated on the primary but fully up to date on a healthy secondary. `dataloss` sub-command displays read-only repositories by default through these versions.
......
doc/administration/gitaly/troubleshooting.md
View file @ 95f43e44
...@@ -374,17 +374,23 @@ Here are common errors and potential causes: ...@@ -374,17 +374,23 @@ Here are common errors and potential causes:
### Determine primary Gitaly node ### Determine primary Gitaly node
To determine the current primary Gitaly node for a specific Praefect node: To determine the primary node of a repository:
- Use the `Shard Primary Election` [Grafana chart](praefect.md#grafana) on the - In GitLab 14.6 and later, use the [`praefect metadata`](praefect.md#retrieve-repository-metadata) subcommand.
[`Gitlab Omnibus - Praefect` dashboard](https://gitlab.com/gitlab-org/grafana-dashboards/-/blob/master/omnibus/praefect.json). - In GitLab 13.12 to GitLab 14.5 with [repository-specific primaries](praefect.md#repository-specific-primary-nodes),
This is recommended. use the [`gitlab:praefect:replicas` Rake task](../raketasks/praefect.md#replica-checksums).
- If you do not have Grafana set up, use the following command on each host of each - With legacy election strategies in GitLab 13.12 and earlier, the primary was the same for all repositories in a virtual storage.
Praefect node: To determine the current primary Gitaly node for a specific virtual storage:
```shell - Use the `Shard Primary Election` [Grafana chart](praefect.md#grafana) on the
curl localhost:9652/metrics | grep gitaly_praefect_primaries` [`Gitlab Omnibus - Praefect` dashboard](https://gitlab.com/gitlab-org/grafana-dashboards/-/blob/master/omnibus/praefect.json).
``` This is recommended.
- If you do not have Grafana set up, use the following command on each host of each
Praefect node:
```shell
curl localhost:9652/metrics | grep gitaly_praefect_primaries`
```
### Check that repositories are in sync ### Check that repositories are in sync
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7