Commit a46dfff8 authored by Achilleas Pipinellis's avatar Achilleas Pipinellis

Merge branch 'bbodenmiller-master-patch-63776' into 'master'

Merge integrity check Rake task & doctor docs

See merge request gitlab-org/gitlab!77505
parents dfa2694d 0373b085
......@@ -201,6 +201,84 @@ The LDAP check Rake task tests the bind DN and password credentials
executed as part of the `gitlab:check` task, but can run independently.
See [LDAP Rake Tasks - LDAP Check](ldap.md#check) for details.
## Verify database values can be decrypted using the current secrets
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/20069) in GitLab 13.1.
This task runs through all possible encrypted values in the
database, verifying that they are decryptable using the current
secrets file (`gitlab-secrets.json`).
Automatic resolution is not yet implemented. If you have values that
cannot be decrypted, you can follow steps to reset them, see our
docs on what to do [when the secrets file is lost](../../raketasks/backup_restore.md#when-the-secrets-file-is-lost).
This can take a very long time, depending on the size of your
database, as it checks all rows in all tables.
**Omnibus Installation**
```shell
sudo gitlab-rake gitlab:doctor:secrets
```
**Source Installation**
```shell
bundle exec rake gitlab:doctor:secrets RAILS_ENV=production
```
**Example output**
```plaintext
I, [2020-06-11T17:17:54.951815 #27148] INFO -- : Checking encrypted values in the database
I, [2020-06-11T17:18:12.677708 #27148] INFO -- : - ApplicationSetting failures: 0
I, [2020-06-11T17:18:12.823692 #27148] INFO -- : - User failures: 0
[...] other models possibly containing encrypted data
I, [2020-06-11T17:18:14.938335 #27148] INFO -- : - Group failures: 1
I, [2020-06-11T17:18:15.559162 #27148] INFO -- : - Operations::FeatureFlagsClient failures: 0
I, [2020-06-11T17:18:15.575533 #27148] INFO -- : - ScimOauthAccessToken failures: 0
I, [2020-06-11T17:18:15.575678 #27148] INFO -- : Total: 1 row(s) affected
I, [2020-06-11T17:18:15.575711 #27148] INFO -- : Done!
```
### Verbose mode
To get more detailed information about which rows and columns can't be
decrypted, you can pass a `VERBOSE` environment variable:
**Omnibus Installation**
```shell
sudo gitlab-rake gitlab:doctor:secrets VERBOSE=1
```
**Source Installation**
```shell
bundle exec rake gitlab:doctor:secrets RAILS_ENV=production VERBOSE=1
```
**Example verbose output**
<!-- vale gitlab.SentenceSpacing = NO -->
```plaintext
I, [2020-06-11T17:17:54.951815 #27148] INFO -- : Checking encrypted values in the database
I, [2020-06-11T17:18:12.677708 #27148] INFO -- : - ApplicationSetting failures: 0
I, [2020-06-11T17:18:12.823692 #27148] INFO -- : - User failures: 0
[...] other models possibly containing encrypted data
D, [2020-06-11T17:19:53.224344 #27351] DEBUG -- : > Something went wrong for Group[10].runners_token: Validation failed: Route can't be blank
I, [2020-06-11T17:19:53.225178 #27351] INFO -- : - Group failures: 1
D, [2020-06-11T17:19:53.225267 #27351] DEBUG -- : - Group[10]: runners_token
I, [2020-06-11T17:18:15.559162 #27148] INFO -- : - Operations::FeatureFlagsClient failures: 0
I, [2020-06-11T17:18:15.575533 #27148] INFO -- : - ScimOauthAccessToken failures: 0
I, [2020-06-11T17:18:15.575678 #27148] INFO -- : Total: 1 row(s) affected
I, [2020-06-11T17:18:15.575711 #27148] INFO -- : Done!
```
<!-- vale gitlab.SentenceSpacing = YES -->
## Troubleshooting
The following are solutions to problems you might discover using the Rake tasks documented
......
---
stage: Enablement
group: Geo
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
redirect_to: 'check.md#verify-database-values-can-be-decrypted-using-the-current-secrets'
remove_date: '2022-03-04'
---
# Doctor Rake tasks **(FREE SELF)**
This document was moved to [another location](check.md#verify-database-values-can-be-decrypted-using-the-current-secrets).
This is a collection of tasks to help investigate and repair
problems caused by data integrity issues.
## Verify database values can be decrypted using the current secrets
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/20069) in GitLab 13.1.
This task runs through all possible encrypted values in the
database, verifying that they are decryptable using the current
secrets file (`gitlab-secrets.json`).
Automatic resolution is not yet implemented. If you have values that
cannot be decrypted, you can follow steps to reset them, see our
docs on what to do [when the secrets file is lost](../../raketasks/backup_restore.md#when-the-secrets-file-is-lost).
This can take a very long time, depending on the size of your
database, as it checks all rows in all tables.
**Omnibus Installation**
```shell
sudo gitlab-rake gitlab:doctor:secrets
```
**Source Installation**
```shell
bundle exec rake gitlab:doctor:secrets RAILS_ENV=production
```
**Example output**
```plaintext
I, [2020-06-11T17:17:54.951815 #27148] INFO -- : Checking encrypted values in the database
I, [2020-06-11T17:18:12.677708 #27148] INFO -- : - ApplicationSetting failures: 0
I, [2020-06-11T17:18:12.823692 #27148] INFO -- : - User failures: 0
[...] other models possibly containing encrypted data
I, [2020-06-11T17:18:14.938335 #27148] INFO -- : - Group failures: 1
I, [2020-06-11T17:18:15.559162 #27148] INFO -- : - Operations::FeatureFlagsClient failures: 0
I, [2020-06-11T17:18:15.575533 #27148] INFO -- : - ScimOauthAccessToken failures: 0
I, [2020-06-11T17:18:15.575678 #27148] INFO -- : Total: 1 row(s) affected
I, [2020-06-11T17:18:15.575711 #27148] INFO -- : Done!
```
### Verbose mode
To get more detailed information about which rows and columns can't be
decrypted, you can pass a `VERBOSE` environment variable:
**Omnibus Installation**
```shell
sudo gitlab-rake gitlab:doctor:secrets VERBOSE=1
```
**Source Installation**
```shell
bundle exec rake gitlab:doctor:secrets RAILS_ENV=production VERBOSE=1
```
**Example verbose output**
<!-- vale gitlab.SentenceSpacing = NO -->
```plaintext
I, [2020-06-11T17:17:54.951815 #27148] INFO -- : Checking encrypted values in the database
I, [2020-06-11T17:18:12.677708 #27148] INFO -- : - ApplicationSetting failures: 0
I, [2020-06-11T17:18:12.823692 #27148] INFO -- : - User failures: 0
[...] other models possibly containing encrypted data
D, [2020-06-11T17:19:53.224344 #27351] DEBUG -- : > Something went wrong for Group[10].runners_token: Validation failed: Route can't be blank
I, [2020-06-11T17:19:53.225178 #27351] INFO -- : - Group failures: 1
D, [2020-06-11T17:19:53.225267 #27351] DEBUG -- : - Group[10]: runners_token
I, [2020-06-11T17:18:15.559162 #27148] INFO -- : - Operations::FeatureFlagsClient failures: 0
I, [2020-06-11T17:18:15.575533 #27148] INFO -- : - ScimOauthAccessToken failures: 0
I, [2020-06-11T17:18:15.575678 #27148] INFO -- : Total: 1 row(s) affected
I, [2020-06-11T17:18:15.575711 #27148] INFO -- : Done!
```
<!-- vale gitlab.SentenceSpacing = YES -->
<!-- This redirect file can be deleted after 2022-03-04. -->
<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
......@@ -120,6 +120,8 @@ You may also have a look at our troubleshooting guides for:
- [GitLab](../index.md#troubleshooting)
- [Omnibus GitLab](https://docs.gitlab.com/omnibus/index.html#troubleshooting)
Additionally you should also [verify database values can be decrypted using the current secrets](check.md#verify-database-values-can-be-decrypted-using-the-current-secrets).
To run `gitlab:check`, run:
**Omnibus Installation**
......
......@@ -526,7 +526,7 @@ master f05321a5b5728bd8a89b7bf530aa44043c951dce...7d02e575fd790e76
### Find mirrors with "bad decrypt" errors
This content has been converted to a Rake task, see the [Doctor Rake tasks docs](../raketasks/doctor.md).
This content has been converted to a Rake task, see [verify database values can be decrypted using the current secrets](../raketasks/check.md#verify-database-values-can-be-decrypted-using-the-current-secrets).
### Transfer mirror users and tokens to a single service account
......@@ -1073,7 +1073,7 @@ area on disk. It remains to be seen exactly how or whether the deletion is usefu
### Bad Decrypt Script (for encrypted variables)
This content has been converted to a Rake task, see the [Doctor Rake tasks docs](../raketasks/doctor.md).
This content has been converted to a Rake task, see [verify database values can be decrypted using the current secrets](../raketasks/check.md#verify-database-values-can-be-decrypted-using-the-current-secrets).
As an example of repairing, if `ProjectImportData Bad count:` is detected and the decision is made to delete the
encrypted credentials to allow manual reentry:
......@@ -1116,7 +1116,7 @@ gitlab-rails runner /tmp/encrypted-tokens.rb
### Decrypt Script for encrypted tokens
This content has been converted to a Rake task, see the [Doctor Rake tasks docs](../raketasks/doctor.md).
This content has been converted to a Rake task, see [verify database values can be decrypted using the current secrets](../raketasks/check.md#verify-database-values-can-be-decrypted-using-the-current-secrets).
## Geo
......
......@@ -990,7 +990,7 @@ sudo gitlab-ctl restart
sudo gitlab-rake gitlab:check SANITIZE=true
```
In GitLab 13.1 and later, check [database values can be decrypted](../administration/raketasks/doctor.md)
In GitLab 13.1 and later, check [database values can be decrypted](../administration/raketasks/check.md#verify-database-values-can-be-decrypted-using-the-current-secrets)
especially if `/etc/gitlab/gitlab-secrets.json` was restored, or if a different server is
the target for the restore.
......@@ -1359,8 +1359,8 @@ Use the information in the following sections at your own risk.
#### Verify that all values can be decrypted
You can determine if your database contains values that can't be decrypted by using the
[Secrets Doctor Rake task](../administration/raketasks/doctor.md).
You can determine if your database contains values that can't be decrypted by using a
[Rake task](../administration/raketasks/check.md#verify-database-values-can-be-decrypted-using-the-current-secrets).
#### Take a backup
......
......@@ -26,7 +26,6 @@ The following Rake tasks are available for use with GitLab:
| [Back up and restore](backup_restore.md) | Back up, restore, and migrate GitLab instances between servers. |
| [Clean up](cleanup.md) | Clean up unneeded items from GitLab instances. |
| [Development](../development/rake_tasks.md) | Tasks for GitLab contributors. |
| [Doctor tasks](../administration/raketasks/doctor.md) | Checks for data integrity issues. |
| [Elasticsearch](../integration/elasticsearch.md#gitlab-advanced-search-rake-tasks) | Maintain Elasticsearch in a GitLab instance. |
| [Enable namespaces](features.md) | Enable usernames and namespaces for user projects. |
| [General maintenance](../administration/raketasks/maintenance.md) | General maintenance and self-check tasks. |
......@@ -34,7 +33,7 @@ The following Rake tasks are available for use with GitLab:
| [GitHub import](../administration/raketasks/github_import.md) | Retrieve and import repositories from GitHub. |
| [Import repositories](import.md) | Import bare repositories into your GitLab instance. |
| [Import large project exports](../development/import_project.md#importing-via-a-rake-task) | Import large GitLab [project exports](../user/project/settings/import_export.md). |
| [Integrity checks](../administration/raketasks/check.md) | Check the integrity of repositories, files, and LDAP. |
| [Integrity checks](../administration/raketasks/check.md) | Check the integrity of repositories, files, LDAP, and more. |
| [LDAP maintenance](../administration/raketasks/ldap.md) | [LDAP](../administration/auth/ldap/index.md)-related tasks. |
| [List repositories](list_repos.md) | List all GitLab-managed Git repositories on disk. |
| [Migrate snippets to Git](migrate_snippets.md) | Migrate GitLab Snippets to Git repositories, and show the migration status. |
......
......@@ -35,7 +35,7 @@ to ensure the major components of GitLab are working:
sudo gitlab-rake gitlab:check
```
1. Confirm that encrypted database values [can be decrypted](../administration/raketasks/doctor.md#verify-database-values-can-be-decrypted-using-the-current-secrets):
1. Confirm that encrypted database values [can be decrypted](../administration/raketasks/check.md#verify-database-values-can-be-decrypted-using-the-current-secrets):
```shell
sudo gitlab-rake gitlab:doctor:secrets
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment