Commit a4dbb3ef authored by Rémy Coutable's avatar Rémy Coutable

Merge branch 'fix/importing-io-timing-issue' into 'master'

Fix timing problems running imports on production

Fixes https://gitlab.com/gitlab-com/infrastructure/issues/151

I've found out that in staging, the imported file is not copied fully by the time sidekiq runs the job, this should hopefully fix it. (Tested against staging).

See merge request !5523
Signed-off-by: default avatarRémy Coutable <remy@rymai.me>
parent 3fbe2a5e
...@@ -2,6 +2,7 @@ Please view this file on the master branch, on stable branches it's out of date. ...@@ -2,6 +2,7 @@ Please view this file on the master branch, on stable branches it's out of date.
v 8.10.3 (unreleased) v 8.10.3 (unreleased)
- Fix Import/Export issue importing milestones and labels not associated properly. !5426 - Fix Import/Export issue importing milestones and labels not associated properly. !5426
- Fix timing problems running imports on production. !5523
v 8.10.2 v 8.10.2
- User can now search branches by name. !5144 - User can now search branches by name. !5144
......
...@@ -3,6 +3,8 @@ module Gitlab ...@@ -3,6 +3,8 @@ module Gitlab
class FileImporter class FileImporter
include Gitlab::ImportExport::CommandLineUtil include Gitlab::ImportExport::CommandLineUtil
MAX_RETRIES = 8
def self.import(*args) def self.import(*args)
new(*args).import new(*args).import
end end
...@@ -14,7 +16,10 @@ module Gitlab ...@@ -14,7 +16,10 @@ module Gitlab
def import def import
FileUtils.mkdir_p(@shared.export_path) FileUtils.mkdir_p(@shared.export_path)
wait_for_archived_file do
decompress_archive decompress_archive
end
rescue => e rescue => e
@shared.error(e) @shared.error(e)
false false
...@@ -22,6 +27,17 @@ module Gitlab ...@@ -22,6 +27,17 @@ module Gitlab
private private
# Exponentially sleep until I/O finishes copying the file
def wait_for_archived_file
MAX_RETRIES.times do |retry_number|
break if File.exist?(@archive_file)
sleep(2**retry_number)
end
yield
end
def decompress_archive def decompress_archive
result = untar_zxf(archive: @archive_file, dir: @shared.export_path) result = untar_zxf(archive: @archive_file, dir: @shared.export_path)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment