Improve more Rake task topics

44a2fc43 · Evan Read · Marcel Amirault · 8b5cd5fe · 44a2fc43 · 44a2fc43
Commit 44a2fc43 authored Apr 28, 2020 by Evan Read Committed by Marcel Amirault Apr 28, 2020
5 changed files
--- a/doc/administration/raketasks/uploads/migrate.md
+++ b/doc/administration/raketasks/uploads/migrate.md
-# Uploads Migrate Rake Tasks
+# Uploads migrate Rake tasks
-## Migrate to Object Storage
+`gitlab:uploads:migrate` migrates uploads between different storage types.
-After [configuring the object storage](../../uploads.md#using-object-storage-core-only) for GitLab's uploads, you may use this task to migrate existing uploads from the local storage to the remote storage.
+## Migrate to object storage
->**Note:**
+After [configuring the object storage](../../uploads.md#using-object-storage-core-only) for GitLab's
-All of the processing will be done in a background worker and requires **no downtime**.
+uploads, use this task to migrate existing uploads from the local storage to the remote storage.
+Read more about using [object storage with GitLab](../../object_storage.md).
-[Read more about using object storage with GitLab](../../object_storage.md).
+NOTE: **Note:**
+All of the processing will be done in a background worker and requires **no downtime**.
 ### All-in-one Rake task
-GitLab provides a wrapper Rake task that migrates all uploaded files - avatars,
+GitLab provides a wrapper Rake task that migrates all uploaded files (for example avatars, logos,
-logos, attachments, favicon, etc. - to object storage in one go. Under the hood,
+attachments, and favicon) to object storage in one step. The wrapper task invokes individual Rake
-it invokes individual Rake tasks to migrate files falling under each of this
+tasks to migrate files falling under each of these categories one by one.
-category one by one. The specifications of these individual Rake tasks are
-described in the next section.
+These [individual Rake tasks](#individual-rake-tasks) are described in the next section.
+To migrate all uploads from local storage to object storage, run:
 **Omnibus Installation**
@@ -31,26 +36,29 @@ sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:migrate:all
 ### Individual Rake tasks
->**Note:**
+If you already ran the [all-in-one Rake task](#all-in-one-rake-task), there is no need to run these
-If you already ran the Rake task mentioned above, no need to run these individual Rake tasks as that has been done automatically.
+individual tasks.
+The Rake task uses three parameters to find uploads to migrate:
-The Rake task uses 3 parameters to find uploads to migrate.
+| Parameter        | Type          | Description                                            |
+|:-----------------|:--------------|:-------------------------------------------------------|
+| `uploader_class` | string        | Type of the uploader to migrate from.                  |
+| `model_class`    | string        | Type of the model to migrate from.                     |
+| `mount_point`    | string/symbol | Name of the model's column the uploader is mounted on. |
-Parameter | Type | Description
+NOTE: **Note:**
--------- | ---- | -----------
+These parameters are mainly internal to GitLab's structure, you may want to refer to the task list
-`uploader_class` | string | Type of the uploader to migrate from
+instead below.
-`model_class` | string | Type of the model to migrate from
-`mount_point` | string/symbol | Name of the model's column on which the uploader is mounted on.
->**Note:**
+This task also accepts an environment variable which you can use to override
-These parameters are mainly internal to GitLab's structure, you may want to refer to the task list instead below.
+the default batch size:
-This task also accepts some environment variables which you can use to override
+| Variable | Type    | Description                                       |
-certain values:
+|:---------|:--------|:--------------------------------------------------|
+| `BATCH`  | integer | Specifies the size of the batch. Defaults to 200. |
-Variable | Type | Description
+The following shows how to run `gitlab:uploads:migrate` for individual types of uploads.
-------- | ---- | -----------
-`BATCH`   | integer  | Specifies the size of the batch. Defaults to 200.
 **Omnibus Installation**
@@ -82,7 +90,7 @@ gitlab-rake "gitlab:uploads:migrate[DesignManagement::DesignV432x230Uploader, De
 **Source Installation**
->**Note:**
+NOTE: **Note:**
 Use `RAILS_ENV=production` for every task.
 ```shell
@@ -111,12 +119,10 @@ sudo -u git -H bundle exec rake "gitlab:uploads:migrate[FileUploader, MergeReque
 sudo -u git -H bundle exec rake "gitlab:uploads:migrate[DesignManagement::DesignV432x230Uploader, DesignManagement::Action]"
 ```
-## Migrate from object storage to local storage
+## Migrate to local storage
-If you need to disable Object Storage for any reason, first you need to migrate
+If you need to disable [object storage](../../object_storage.md) for any reason, you must first
-your data out of Object Storage and back into your local storage.
+migrate your data out of object storage and back into your local storage.
-**Before proceeding, it is important to disable both `direct_upload` and `background_upload` under `uploads` settings in `gitlab.rb`**
 CAUTION: **Warning:**
 **Extended downtime is required** so no new files are created in object storage during
@@ -126,23 +132,29 @@ To follow progress, see the [relevant issue](https://gitlab.com/gitlab-org/gitla
 ### All-in-one Rake task
-GitLab provides a wrapper Rake task that migrates all uploaded files - avatars,
+GitLab provides a wrapper Rake task that migrates all uploaded files (for example, avatars, logos,
-logos, attachments, favicon, etc. - to local storage in one go. Under the hood,
+attachments, and favicon) to local storage in one step. The wrapper task invokes individual Rake
-it invokes individual Rake tasks to migrate files falling under each of this
+tasks to migrate files falling under each of these categories one by one.
-category one by one. For details on these Rake tasks please [refer to the section above](#individual-rake-tasks),
+For details on these Rake tasks, refer to [Individual Rake tasks](#individual-rake-tasks),
 keeping in mind the task name in this case is `gitlab:uploads:migrate_to_local`.
-**Omnibus Installation**
+To migrate uploads from object storage to local storage:
-```shell
+1. Disable both `direct_upload` and `background_upload` under `uploads` settings in `gitlab.rb`.
-gitlab-rake "gitlab:uploads:migrate_to_local:all"
+1. Run the Rake task:
-```
-**Source Installation**
+   **Omnibus Installation**
-```shell
+   ```shell
-sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:migrate_to_local:all
+   gitlab-rake "gitlab:uploads:migrate_to_local:all"
-```
+   ```
+   **Source Installation**
+   ```shell
+   sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:migrate_to_local:all
+   ```
-After this is done, you may disable Object Storage by undoing the changes described
+After running the Rake task, you can disable object storage by undoing the changes described
-in the instructions to [configure object storage](../../uploads.md#using-object-storage-core-only)
+in the instructions to [configure object storage](../../uploads.md#using-object-storage-core-only).
--- a/doc/administration/raketasks/uploads/sanitize.md
+++ b/doc/administration/raketasks/uploads/sanitize.md
-# Uploads Sanitize tasks
+# Uploads sanitize Rake tasks
+Since GitLab 11.9, EXIF data is automatically stripped from JPG or TIFF image uploads.
+EXIF data may contain sensitive information (for example, GPS location), so you
+can remove EXIF data from existing images that were uploaded to an earlier version of GitLab.
 ## Requirements
-You need `exiftool` installed on your system. If you installed GitLab:
+To run this Rake task, you need `exiftool` installed on your system. If you installed GitLab:
 - Using the Omnibus package, you're all set.
 - From source, make sure `exiftool` is installed:
@@ -17,48 +22,48 @@ You need `exiftool` installed on your system. If you installed GitLab:
 ## Remove EXIF data from existing uploads
-Since 11.9 EXIF data are automatically stripped from JPG or TIFF image uploads.
+To remove EXIF data from existing uploads, run the following command:
-Because EXIF data may contain sensitive information (e.g. GPS location), you
-can remove EXIF data also from existing images which were uploaded before
-with the following command:
 ```shell
 sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:sanitize:remove_exif
 ```
-This command by default runs in dry mode and it doesn't remove EXIF data. It can be used for
+By default, this command runs in "dry run" mode and doesn't remove EXIF data. It can be used for
 checking if (and how many) images should be sanitized.
 The Rake task accepts following parameters.
-Parameter | Type | Description
+| Parameter    | Type    | Description                                                                                                                 |
--------- | ---- | -----------
+|:-------------|:--------|:----------------------------------------------------------------------------------------------------------------------------|
-`start_id` | integer | Only uploads with equal or greater ID will be processed
+| `start_id`   | integer | Only uploads with equal or greater ID will be processed                                                                     |
-`stop_id` | integer | Only uploads with equal or smaller ID will be processed
+| `stop_id`    | integer | Only uploads with equal or smaller ID will be processed                                                                     |
-`dry_run` | boolean | Do not remove EXIF data, only check if EXIF data are present or not, default: true
+| `dry_run`    | boolean | Do not remove EXIF data, only check if EXIF data are present or not. Defaults to `true`                                     |
-`sleep_time` | float | Pause for number of seconds after processing each image, default: 0.3 seconds
+| `sleep_time` | float   | Pause for number of seconds after processing each image. Defaults to 0.3 seconds                                            |
-`uploader` | string | Run sanitization only for uploads of the given uploader (`FileUploader`, `PersonalFileUploader`, `NamespaceFileUploader`)
+| `uploader`   | string  | Run sanitization only for uploads of the given uploader: `FileUploader`, `PersonalFileUploader`, or `NamespaceFileUploader` |
-`since` | date | Run sanitization only for uploads newer than given date (e.g. `2019-05-01`)
+| `since`      | date    | Run sanitization only for uploads newer than given date. For example, `2019-05-01`                                          |
-If you have too many uploads, you can speed up sanitization by setting
+If you have too many uploads, you can speed up sanitization by:
-`sleep_time` to a lower value or by running multiple Rake tasks in parallel,
-each with a separate range of upload IDs (by setting `start_id` and `stop_id`).
-To run the command without dry mode and remove EXIF data from all uploads, you can use:
+- Setting `sleep_time` to a lower value.
+- Running multiple Rake tasks in parallel, each with a separate range of upload IDs (by setting
+  `start_id` and `stop_id`).
+To remove EXIF data from all uploads, use:
 ```shell
 sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:sanitize:remove_exif[,,false,] 2>&1 | tee exif.log
 ```
-To run the command without dry mode on uploads with ID between 100 and 5000 and pause for 0.1 second, you can use:
+To remove EXIF data on uploads with an ID between 100 and 5000 and pause for 0.1 second after each file, use:
 ```shell
 sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:sanitize:remove_exif[100,5000,false,0.1] 2>&1 | tee exif.log
 ```
-Because the output of commands will be probably long, the output is written also into exif.log file.
+The output is written into an `exif.log` file because it will probably be long.
+If sanitization fails for an upload, an error message should be in the output of the Rake task.
+Typical reasons include that the file is missing in the storage or it's not a valid image.
-If sanitization fails for an upload, an error message should be in the output of the Rake task (typical reasons may
+[Report](https://gitlab.com/gitlab-org/gitlab/issues/new) any issues and use the prefix 'EXIF' in
-be that the file is missing in the storage or it's not a valid image). Please
+the issue title with the error output and (if possible) the image.
-[report](https://gitlab.com/gitlab-org/gitlab-foss/issues/new) any issues at `gitlab.com` and use
-prefix 'EXIF' in issue title with the error output and (if possible) the image.
--- a/doc/raketasks/user_management.md
+++ b/doc/raketasks/user_management.md
 # User management
+GitLab provides Rake tasks for user management.
 ## Add user as a developer to all projects
+To add a user as a developer to all projects, run:
 ```shell
 # omnibus-gitlab
 sudo gitlab-rake gitlab:import:user_to_projects[username@domain.tld]
@@ -12,9 +16,7 @@ bundle exec rake gitlab:import:user_to_projects[username@domain.tld] RAILS_ENV=p
 ## Add all users to all projects
-Notes:
+To add all users to all projects, run:
- admin users are added as maintainers
 ```shell
 # omnibus-gitlab
@@ -24,8 +26,13 @@ sudo gitlab-rake gitlab:import:all_users_to_all_projects
 bundle exec rake gitlab:import:all_users_to_all_projects RAILS_ENV=production
 ```
+NOTE: **Note:**
+Admin users are added as maintainers.
 ## Add user as a developer to all groups
+To add a user as a developer to all groups, run:
 ```shell
 # omnibus-gitlab
 sudo gitlab-rake gitlab:import:user_to_groups[username@domain.tld]
@@ -36,9 +43,7 @@ bundle exec rake gitlab:import:user_to_groups[username@domain.tld] RAILS_ENV=pro
 ## Add all users to all groups
-Notes:
+To add all users to all groups, run:
- admin users are added as owners so they can add additional users to the group
 ```shell
 # omnibus-gitlab
@@ -48,19 +53,25 @@ sudo gitlab-rake gitlab:import:all_users_to_all_groups
 bundle exec rake gitlab:import:all_users_to_all_groups RAILS_ENV=production
 ```
-## Maintain tight control over the number of active users on your GitLab installation
+NOTE: **Note:**
+Admin users are added as owners so they can add additional users to the group.
+## Control the number of active users
- Enable this setting to keep new users blocked until they have been cleared by the admin (default: false).
+Enable this setting to keep new users blocked until they have been cleared by the administrator.
+Defaults to `false`:
 ```plaintext
 block_auto_created_users: false
 ```
-## Disable Two-factor Authentication (2FA) for all users
+## Disable two-factor authentication for all users
-This task will disable 2FA for all users that have it enabled. This can be
+This task disables two-factor authentication (2FA) for all users that have it enabled. This can be
 useful if GitLab's `config/secrets.yml` file has been lost and users are unable
-to login, for example.
+to log in, for example.
+To disable two-factor authentication for all users, run:
 ```shell
 # omnibus-gitlab
@@ -70,65 +81,65 @@ sudo gitlab-rake gitlab:two_factor:disable_for_all_users
 bundle exec rake gitlab:two_factor:disable_for_all_users RAILS_ENV=production
 ```
-## Rotate Two-factor Authentication (2FA) encryption key
+## Rotate two-factor authentication encryption key
-GitLab stores the secret data enabling 2FA to work in an encrypted database
+GitLab stores the secret data required for two-factor authentication (2FA) in an encrypted
-column. The encryption key for this data is known as `otp_key_base`, and is
+database column. The encryption key for this data is known as `otp_key_base`, and is
 stored in `config/secrets.yml`.
 If that file is leaked, but the individual 2FA secrets have not, it's possible
 to re-encrypt those secrets with a new encryption key. This allows you to change
 the leaked key without forcing all users to change their 2FA details.
-First, look up the old key. This is in the `config/secrets.yml` file, but
+To rotate the two-factor authentication encryption key:
-**make sure you're working with the production section**. The line you're
-interested in will look like this:
-```yaml
+1. Look up the old key. This is in the `config/secrets.yml` file, but **make sure you're working
-production:
+   with the production section**. The line you're interested in will look like this:
-  otp_key_base: ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
-```
-Next, generate a new secret:
+   ```yaml
+   production:
+     otp_key_base: ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
+   ```
-```shell
+1. Generate a new secret:
-# omnibus-gitlab
-sudo gitlab-rake secret
-# installation from source
+   ```shell
-bundle exec rake secret RAILS_ENV=production
+   # omnibus-gitlab
-```
+   sudo gitlab-rake secret
-Now you need to stop the GitLab server, back up the existing secrets file and
+   # installation from source
-update the database:
+   bundle exec rake secret RAILS_ENV=production
+   ```
-```shell
+1. Stop the GitLab server, back up the existing secrets file, and update the database:
-# omnibus-gitlab
-sudo gitlab-ctl stop
-sudo cp config/secrets.yml config/secrets.yml.bak
-sudo gitlab-rake gitlab:two_factor:rotate_key:apply filename=backup.csv old_key=<old key> new_key=<new key>
-# installation from source
+   ```shell
-sudo /etc/init.d/gitlab stop
+   # omnibus-gitlab
-cp config/secrets.yml config/secrets.yml.bak
+   sudo gitlab-ctl stop
-bundle exec rake gitlab:two_factor:rotate_key:apply filename=backup.csv old_key=<old key> new_key=<new key> RAILS_ENV=production
+   sudo cp config/secrets.yml config/secrets.yml.bak
-```
+   sudo gitlab-rake gitlab:two_factor:rotate_key:apply filename=backup.csv old_key=<old key> new_key=<new key>
-The `<old key>` value can be read from `config/secrets.yml`; `<new key>` was
+   # installation from source
-generated earlier. The **encrypted** values for the user 2FA secrets will be
+   sudo /etc/init.d/gitlab stop
-written to the specified `filename` - you can use this to rollback in case of
+   cp config/secrets.yml config/secrets.yml.bak
-error.
+   bundle exec rake gitlab:two_factor:rotate_key:apply filename=backup.csv old_key=<old key> new_key=<new key> RAILS_ENV=production
+   ```
-Finally, change `config/secrets.yml` to set `otp_key_base` to `<new key>` and
+   The `<old key>` value can be read from `config/secrets.yml` (`<new key>` was
-restart. Again, make sure you're operating in the **production** section.
+   generated earlier). The **encrypted** values for the user 2FA secrets will be
+   written to the specified `filename`. You can use this to rollback in case of
+   error.
-```shell
+1. Change `config/secrets.yml` to set `otp_key_base` to `<new key>` and restart. Again, make sure
-# omnibus-gitlab
+   you're operating in the **production** section.
-sudo gitlab-ctl start
-# installation from source
+   ```shell
-sudo /etc/init.d/gitlab start
+   # omnibus-gitlab
-```
+   sudo gitlab-ctl start
+   # installation from source
+   sudo /etc/init.d/gitlab start
+   ```
 If there are any problems (perhaps using the wrong value for `old_key`), you can
 restore your backup of `config/secrets.yml` and rollback the changes:

--- a/doc/raketasks/web_hooks.md
+++ b/doc/raketasks/web_hooks.md
 # Webhooks administration **(CORE ONLY)**
-## Add a webhook for **ALL** projects
+GitLab provides Rake tasks for webhooks management.
+Requests to the [local network by webhooks](../security/webhooks.md) can be allowed or blocked by an
+administrator.
+## Add a webhook to all projects
+To add a webhook to all projects, run:
 ```shell
 # omnibus-gitlab
 sudo gitlab-rake gitlab:web_hook:add URL="http://example.com/hook"
 # source installations
 bundle exec rake gitlab:web_hook:add URL="http://example.com/hook" RAILS_ENV=production
 ```
-## Add a webhook for projects in a given **NAMESPACE**
+## Add a webhook to projects in a namespace
+To add a webhook to all projects in a specific namespace, run:
 ```shell
 # omnibus-gitlab
-sudo gitlab-rake gitlab:web_hook:add URL="http://example.com/hook" NAMESPACE=acme
+sudo gitlab-rake gitlab:web_hook:add URL="http://example.com/hook" NAMESPACE=<namespace>
 # source installations
-bundle exec rake gitlab:web_hook:add URL="http://example.com/hook" NAMESPACE=acme RAILS_ENV=production
+bundle exec rake gitlab:web_hook:add URL="http://example.com/hook" NAMESPACE=<namespace> RAILS_ENV=production
 ```
-## Remove a webhook from **ALL** projects using
+## Remove a webhook from projects
+To remove a webhook from all projects, run:
 ```shell
 # omnibus-gitlab
 sudo gitlab-rake gitlab:web_hook:rm URL="http://example.com/hook"
 # source installations
 bundle exec rake gitlab:web_hook:rm URL="http://example.com/hook" RAILS_ENV=production
 ```
-## Remove a webhook from projects in a given **NAMESPACE**
+## Remove a webhook from projects in a namespace
+To remove a webhook from projects in a specific namespace, run:
 ```shell
 # omnibus-gitlab
-sudo gitlab-rake gitlab:web_hook:rm URL="http://example.com/hook" NAMESPACE=acme
+sudo gitlab-rake gitlab:web_hook:rm URL="http://example.com/hook" NAMESPACE=<namespace>
 # source installations
-bundle exec rake gitlab:web_hook:rm URL="http://example.com/hook" NAMESPACE=acme RAILS_ENV=production
+bundle exec rake gitlab:web_hook:rm URL="http://example.com/hook" NAMESPACE=<namespace> RAILS_ENV=production
 ```
-## List **ALL** webhooks
+## List all webhooks
+To list all webhooks, run:
 ```shell
 # omnibus-gitlab
 sudo gitlab-rake gitlab:web_hook:list
 # source installations
 bundle exec rake gitlab:web_hook:list RAILS_ENV=production
 ```
-## List the webhooks from projects in a given **NAMESPACE**
+## List webhooks for projects in a namespace
+To list all webhook for projects in a specified namespace, run:
 ```shell
 # omnibus-gitlab
-sudo gitlab-rake gitlab:web_hook:list NAMESPACE=acme
+sudo gitlab-rake gitlab:web_hook:list NAMESPACE=<namespace>
 # source installations
-bundle exec rake gitlab:web_hook:list NAMESPACE=acme RAILS_ENV=production
+bundle exec rake gitlab:web_hook:list NAMESPACE=<namespace> RAILS_ENV=production
 ```
-## Local requests in webhooks
-[Requests to local network by webhooks](../security/webhooks.md) can be allowed
-or blocked by an administrator.
--- a/doc/raketasks/x509_signatures.md
+++ b/doc/raketasks/x509_signatures.md
 # X509 signatures
-When [signing commits with x509](../user/project/repository/x509_signed_commits/index.md)
+When [signing commits with x509](../user/project/repository/x509_signed_commits/index.md),
-the trust anchor might change and the signatures stored within the database have
+the trust anchor might change and the signatures stored within the database must be updated.
-to be updated.
 ## Update all x509 signatures
-This task loops through all X509 signed commits and updates their verification
+This task loops through all X509 signed commits and updates their verification based on current
-based on current certificate store.
+certificate store.
+To update all x509 signatures, run:
 **Omnibus Installation**