Merge branch 'fix/importing-io-timing-issue' into 'master'

Fix timing problems running imports on production

Fixes https://gitlab.com/gitlab-com/infrastructure/issues/151

I've found out that in staging, the imported file is not copied fully by the time sidekiq runs the job, this should hopefully fix it. (Tested against staging).

See merge request !5523
Signed-off-by: Rémy Coutable <remy@rymai.me>

CHANGELOG

@@ -2,6 +2,7 @@ Please view this file on the master branch, on stable branches it's out of date.
 
 v 8.10.3 (unreleased)
   - Fix Import/Export issue importing milestones and labels not associated properly. !5426
+  - Fix timing problems running imports on production. !5523
 
 v 8.10.2
   - User can now search branches by name. !5144

lib/gitlab/import_export/file_importer.rb

@@ -3,6 +3,8 @@ module Gitlab
     class FileImporter
       include Gitlab::ImportExport::CommandLineUtil
 
+      MAX_RETRIES = 8
+
       def self.import(*args)
         new(*args).import
       end
@@ -14,7 +16,10 @@ module Gitlab
 
       def import
         FileUtils.mkdir_p(@shared.export_path)
+
+        wait_for_archived_file do
           decompress_archive
+        end
       rescue => e
         @shared.error(e)
         false
@@ -22,6 +27,17 @@ module Gitlab
 
       private
 
+      # Exponentially sleep until I/O finishes copying the file
+      def wait_for_archived_file
+        MAX_RETRIES.times do |retry_number|
+          break if File.exist?(@archive_file)
+
+          sleep(2**retry_number)
+        end
+
+        yield
+      end
+
       def decompress_archive
         result = untar_zxf(archive: @archive_file, dir: @shared.export_path)