-
Dylan Griffith authored
This is the last step in https://gitlab.com/groups/gitlab-org/-/epics/5468 to move merge requests out of the main monolithic Elasticsearch index into a dedicated index. This will allow searches to be much faster. This MR contains a new migration which copies all the merge requests from the main index into the new index. Then there is also code which checks if this new migration is completed and from there on sends all reads and writes to the new index. Since it's tricky to write a zero downtime migration this migration will pause indexing while it is running. The expected time taken can be estimated as: 1. We have 18M merge requests in the index 2. The migration breaks out the reindexing into 120 slices (this is how many shards we have in the main index) 3. The slices are executed 1 at a time with a wait time of 1 minute between starting the slice and checking if it is done 4. So best case is 1 slice per 2 minutes => `240 minutes` = `4 hours` total run time 5. Based on some [internal slack thread](https://gitlab.slack.com/archives/C3TMLK465/p1619679775072500) we tracked the similar notes migration we did in the past and it took `9 hours` to run. There are less MRs in the index so we should expect this to take less than 9 hours. The code in this MR is almost identical to what we did for notes in https://gitlab.com/gitlab-org/gitlab/-/merge_requests/53013 including the small fix made in https://gitlab.com/gitlab-org/gitlab/-/merge_requests/57248 and then an extra change to accomodate the new ability to update index settings per type https://gitlab.com/gitlab-org/gitlab/-/merge_requests/57567 Changelog: performance
b4296d0d