Commit ab3dd9a1 authored by Rémy Coutable's avatar Rémy Coutable

Merge branch 'fix/importing-io-timing-issue' into 'master'

Fix timing problems running imports on production

Fixes https://gitlab.com/gitlab-com/infrastructure/issues/151

I've found out that in staging, the imported file is not copied fully by the time sidekiq runs the job, this should hopefully fix it. (Tested against staging).

See merge request !5523
parents be7ab1f2 7e77b1fd
...@@ -3,6 +3,8 @@ module Gitlab ...@@ -3,6 +3,8 @@ module Gitlab
class FileImporter class FileImporter
include Gitlab::ImportExport::CommandLineUtil include Gitlab::ImportExport::CommandLineUtil
MAX_RETRIES = 8
def self.import(*args) def self.import(*args)
new(*args).import new(*args).import
end end
...@@ -14,7 +16,10 @@ module Gitlab ...@@ -14,7 +16,10 @@ module Gitlab
def import def import
FileUtils.mkdir_p(@shared.export_path) FileUtils.mkdir_p(@shared.export_path)
decompress_archive
wait_for_archived_file do
decompress_archive
end
rescue => e rescue => e
@shared.error(e) @shared.error(e)
false false
...@@ -22,6 +27,17 @@ module Gitlab ...@@ -22,6 +27,17 @@ module Gitlab
private private
# Exponentially sleep until I/O finishes copying the file
def wait_for_archived_file
MAX_RETRIES.times do |retry_number|
break if File.exist?(@archive_file)
sleep(2**retry_number)
end
yield
end
def decompress_archive def decompress_archive
result = untar_zxf(archive: @archive_file, dir: @shared.export_path) result = untar_zxf(archive: @archive_file, dir: @shared.export_path)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment