Commit acd8500b authored by Dylan Griffith's avatar Dylan Griffith

Rename background/Elasticsearch migration to Advanced Search migrations

This is to use more consistent language and Advanced Search migration
is clearer about the specific scope.
parent 9d6af80c
...@@ -36,7 +36,7 @@ The type of problem will determine what steps to take. The possible troubleshoot ...@@ -36,7 +36,7 @@ The type of problem will determine what steps to take. The possible troubleshoot
- Indexing. - Indexing.
- Integration. - Integration.
- Performance. - Performance.
- Background Migrations. - Advanced Search Migrations.
### Search Results workflow ### Search Results workflow
...@@ -148,7 +148,7 @@ graph TD; ...@@ -148,7 +148,7 @@ graph TD;
F7(Escalate to<br>GitLab support.) F7(Escalate to<br>GitLab support.)
``` ```
### Background Migrations workflow ### Advanced Search Migrations workflow
```mermaid ```mermaid
graph TD; graph TD;
...@@ -180,7 +180,7 @@ Most Elasticsearch troubleshooting can be broken down into 4 categories: ...@@ -180,7 +180,7 @@ Most Elasticsearch troubleshooting can be broken down into 4 categories:
- [Troubleshooting indexing](#troubleshooting-indexing) - [Troubleshooting indexing](#troubleshooting-indexing)
- [Troubleshooting integration](#troubleshooting-integration) - [Troubleshooting integration](#troubleshooting-integration)
- [Troubleshooting performance](#troubleshooting-performance) - [Troubleshooting performance](#troubleshooting-performance)
- [Troubleshooting background migrations](#troubleshooting-background-migrations) - [Troubleshooting Advanced Search migrations](#troubleshooting-advanced-search-migrations)
Generally speaking, if it does not fall into those four categories, it is either: Generally speaking, if it does not fall into those four categories, it is either:
...@@ -356,15 +356,16 @@ dig further into these. ...@@ -356,15 +356,16 @@ dig further into these.
Feel free to reach out to GitLab support, but this is likely to be something a skilled Feel free to reach out to GitLab support, but this is likely to be something a skilled
Elasticsearch administrator has more experience with. Elasticsearch administrator has more experience with.
### Troubleshooting background migrations ### Troubleshooting Advanced Search migrations
Troubleshooting background migration failures can be difficult and may require contacting Troubleshooting Advanced Search migration failures can be difficult and may
an Elasticsearch administrator or GitLab Support. require contacting an Elasticsearch administrator or GitLab Support.
The best place to start while debugging issues with a background migration is the The best place to start while debugging issues with an Advanced Search
[`elasticsearch.log` file](../logs.md#elasticsearchlog). Migrations will migration is the [`elasticsearch.log` file](../logs.md#elasticsearchlog).
print information while a migration is in progress and any errors encountered. Migrations will log information while a migration is in progress and any
Apply fixes for any errors found in the log and retry the migration. errors encountered. Apply fixes for any errors found in the log and retry
the migration.
If you still encounter issues after retrying the migration, reach out to GitLab support. If you still encounter issues after retrying the migration, reach out to GitLab support.
......
...@@ -249,12 +249,12 @@ end ...@@ -249,12 +249,12 @@ end
### Multi-version compatibility ### Multi-version compatibility
These Elasticsearch migrations, like any other GitLab changes, need to support the case where These Advanced Search migrations, like any other GitLab changes, need to support the case where
[multiple versions of the application are running at the same time](multi_version_compatibility.md). [multiple versions of the application are running at the same time](multi_version_compatibility.md).
Depending on the order of deployment, it's possible that the migration Depending on the order of deployment, it's possible that the migration
has started or finished and there's still a server running the application code from before the has started or finished and there's still a server running the application code from before the
migration. We need to take this into consideration until we can [ensure all Elasticsearch migrations migration. We need to take this into consideration until we can [ensure all Advanced Search migrations
start after the deployment has finished](https://gitlab.com/gitlab-org/gitlab/-/issues/321619). start after the deployment has finished](https://gitlab.com/gitlab-org/gitlab/-/issues/321619).
### Reverting a migration ### Reverting a migration
...@@ -268,7 +268,7 @@ some data is moved) to a later merge request after the migrations have ...@@ -268,7 +268,7 @@ some data is moved) to a later merge request after the migrations have
completed successfully. To be safe, for self-managed customers we should also completed successfully. To be safe, for self-managed customers we should also
defer it to another release if there is risk of important data loss. defer it to another release if there is risk of important data loss.
### Best practices for Elasticsearch migrations ### Best practices for Advanced Search migrations
Follow these best practices for best results: Follow these best practices for best results:
...@@ -286,11 +286,11 @@ Follow these best practices for best results: ...@@ -286,11 +286,11 @@ Follow these best practices for best results:
- Consider adding a retry limit if there is potential for the migration to fail. - Consider adding a retry limit if there is potential for the migration to fail.
This ensures that migrations can be halted if an issue occurs. This ensures that migrations can be halted if an issue occurs.
## Deleting Elasticsearch migrations in a major version upgrade ## Deleting Advanced Search migrations in a major version upgrade
Since our Elasticsearch migrations usually require us to support multiple code Since our Advanced Search migrations usually require us to support multiple
paths for a long period of time it's important to clean those up when we safely code paths for a long period of time it's important to clean those up when we
can. safely can.
We choose to use GitLab major version upgrades as a safe time to remove We choose to use GitLab major version upgrades as a safe time to remove
backwards compatibility for indices that have not been fully migrated. We backwards compatibility for indices that have not been fully migrated. We
...@@ -298,7 +298,7 @@ backwards compatibility for indices that have not been fully migrated. We ...@@ -298,7 +298,7 @@ backwards compatibility for indices that have not been fully migrated. We
documentation](../update/index.md#upgrading-to-a-new-major-version). We also documentation](../update/index.md#upgrading-to-a-new-major-version). We also
choose to remove the migration code and tests so that: choose to remove the migration code and tests so that:
1. We don't need to maintain any code that is called from our Elasticsearch 1. We don't need to maintain any code that is called from our Advanced Search
migrations migrations
1. We don't waste CI time running tests for migrations that we don't support 1. We don't waste CI time running tests for migrations that we don't support
anymore anymore
...@@ -312,8 +312,8 @@ GitLab.com to `%14.0` before the migrations in `%13.11` were finished. Since ...@@ -312,8 +312,8 @@ GitLab.com to `%14.0` before the migrations in `%13.11` were finished. Since
our deployments to GitLab.com are automated and we currently don't have our deployments to GitLab.com are automated and we currently don't have
automated checks to prevent this the extra precaution is warranted. automated checks to prevent this the extra precaution is warranted.
Additionally, even if we did have automated checks to prevent it, we wouldn't Additionally, even if we did have automated checks to prevent it, we wouldn't
actually want to hold up GitLab.com deployments on Elasticsearch migrations as actually want to hold up GitLab.com deployments on Advanced Search migrations
they may still have another week to go and that's too long to block as they may still have another week to go and that's too long to block
deployments. deployments.
### Process for removing migrations ### Process for removing migrations
...@@ -326,7 +326,7 @@ being upgraded to we do the following: ...@@ -326,7 +326,7 @@ being upgraded to we do the following:
```ruby ```ruby
def migrate def migrate
raise "Migration has been deleted in the last major version upgrade." \ log_raise "Migration has been deleted in the last major version upgrade." \
"Migrations are supposed to be finished before upgrading major version https://docs.gitlab.com/ee/update/#upgrading-to-a-new-major-version ." \ "Migrations are supposed to be finished before upgrading major version https://docs.gitlab.com/ee/update/#upgrading-to-a-new-major-version ." \
"In order to correct this issue you will need to reacreate your index from scratch https://docs.gitlab.com/ee/integration/elasticsearch.html#last-resort-to-recreate-an-index ." "In order to correct this issue you will need to reacreate your index from scratch https://docs.gitlab.com/ee/integration/elasticsearch.html#last-resort-to-recreate-an-index ."
end end
......
...@@ -344,7 +344,7 @@ Sometimes, you might want to abandon the unfinished reindex job and resume the i ...@@ -344,7 +344,7 @@ Sometimes, you might want to abandon the unfinished reindex job and resume the i
1. Uncheck the "Pause Elasticsearch indexing" checkbox in **Admin Area > Settings > Advanced Search**. 1. Uncheck the "Pause Elasticsearch indexing" checkbox in **Admin Area > Settings > Advanced Search**.
## Background migrations ## Advanced Search migrations
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/234046) in GitLab 13.6. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/234046) in GitLab 13.6.
...@@ -352,7 +352,7 @@ With reindex migrations running in the background, there's no need for a manual ...@@ -352,7 +352,7 @@ With reindex migrations running in the background, there's no need for a manual
intervention. This usually happens in situations where new features are added to intervention. This usually happens in situations where new features are added to
Advanced Search, which means adding or changing the way content is indexed. Advanced Search, which means adding or changing the way content is indexed.
To confirm that the background migrations ran, you can check with: To confirm that the Advanced Search migrations ran, you can check with:
```shell ```shell
curl "$CLUSTER_URL/gitlab-production-migrations/_search?q=*" | jq . curl "$CLUSTER_URL/gitlab-production-migrations/_search?q=*" | jq .
...@@ -406,7 +406,7 @@ If there appears to be no way you can get the migration to succeed you may ...@@ -406,7 +406,7 @@ If there appears to be no way you can get the migration to succeed you may
consider the [last resort to recreate the index from consider the [last resort to recreate the index from
scratch](#last-resort-to-recreate-an-index). This may allow you to skip over scratch](#last-resort-to-recreate-an-index). This may allow you to skip over
the problem because a newly created index will skip all migrations as the index the problem because a newly created index will skip all migrations as the index
will be recreated with the correct up to date schema. will be recreated with the correct up-to-date schema.
### All migrations must be finished before doing a major upgrade ### All migrations must be finished before doing a major upgrade
......
...@@ -138,16 +138,16 @@ pending_job_classes = scheduled_queue.select { |job| job["class"] == "Background ...@@ -138,16 +138,16 @@ pending_job_classes = scheduled_queue.select { |job| job["class"] == "Background
pending_job_classes.each { |job_class| Gitlab::BackgroundMigration.steal(job_class) } pending_job_classes.each { |job_class| Gitlab::BackgroundMigration.steal(job_class) }
``` ```
## Checking for pending Elasticsearch migrations ## Checking for pending Advanced Search migrations
This section is only applicable if you have enabled the [Elasticsearch This section is only applicable if you have enabled the [Elasticsearch
integration](../integration/elasticsearch.md). integration](../integration/elasticsearch.md).
Major releases require all [Elasticsearch Major releases require all [Advanced Search
migrations](../integration/elasticsearch.md#background-migrations) to be migrations](../integration/elasticsearch.md#advanced-search-migrations) to
finished from the most recent minor release in your current version before the be finished from the most recent minor release in your current version
major version upgrade. You can find pending migrations by running the following before the major version upgrade. You can find pending migrations by
command: running the following command:
**For Omnibus installations** **For Omnibus installations**
...@@ -162,7 +162,7 @@ cd /home/git/gitlab ...@@ -162,7 +162,7 @@ cd /home/git/gitlab
sudo -u git -H bundle exec rake gitlab:elastic:list_pending_migrations sudo -u git -H bundle exec rake gitlab:elastic:list_pending_migrations
``` ```
### What do I do if my Elasticsearch migrations are stuck? ### What do I do if my Advanced Search migrations are stuck?
See [how to retry a halted See [how to retry a halted
migration](../integration/elasticsearch.md#retry-a-halted-migration). migration](../integration/elasticsearch.md#retry-a-halted-migration).
...@@ -212,9 +212,9 @@ before upgrading to a new major version. To see the current size of the `backgro ...@@ -212,9 +212,9 @@ before upgrading to a new major version. To see the current size of the `backgro
If you have enabled the [Elasticsearch If you have enabled the [Elasticsearch
integration](../integration/elasticsearch.md) then you will also need to ensure integration](../integration/elasticsearch.md) then you will also need to ensure
all Elasticsearch migrations are completed in the last minor version within all Advanced Search migrations are completed in the last minor version within
your current version. Be sure to [check for pending Elasticsearch your current version. Be sure to [check for pending Advanced Search
migrations](#checking-for-pending-elasticsearch-migrations) before proceeding migrations](#checking-for-pending-advanced-search-migrations) before proceeding
with the major version upgrade. with the major version upgrade.
If your GitLab instance has any runners associated with it, it is very If your GitLab instance has any runners associated with it, it is very
......
...@@ -28,7 +28,7 @@ ...@@ -28,7 +28,7 @@
.gl-alert-body .gl-alert-body
%h4.gl-alert-title= _('There is a halted Elasticsearch migration') %h4.gl-alert-title= _('There is a halted Elasticsearch migration')
= html_escape_once(_('Check the elasticsearch.log file to debug why the migration was halted and make any changes before retrying the migration. When you fix the cause of the failure, click "Retry migration", and the migration will be scheduled to be retried in the background.')).html_safe = html_escape_once(_('Check the elasticsearch.log file to debug why the migration was halted and make any changes before retrying the migration. When you fix the cause of the failure, click "Retry migration", and the migration will be scheduled to be retried in the background.')).html_safe
= link_to _('Learn more.'), help_page_path('integration/elasticsearch', anchor: 'background-migrations') = link_to _('Learn more.'), help_page_path('integration/elasticsearch', anchor: 'advanced-search-migrations')
.gl-alert-actions .gl-alert-actions
- migration = Elastic::DataMigrationService.halted_migration - migration = Elastic::DataMigrationService.halted_migration
= link_to _('Retry migration'), admin_elasticsearch_retry_migration_path(version: migration.version), class: 'btn gl-alert-action btn-warning gl-button', disabled: @elasticsearch_reindexing_task&.in_progress?, data: { confirm: _('Are you sure you want to retry this migration?') }, method: :post = link_to _('Retry migration'), admin_elasticsearch_retry_migration_path(version: migration.version), class: 'btn gl-alert-action btn-warning gl-button', disabled: @elasticsearch_reindexing_task&.in_progress?, data: { confirm: _('Are you sure you want to retry this migration?') }, method: :post
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment