1. 19 Sep, 2019 1 commit
    • Stan Hu's avatar
      Fix project exports clobbering concurrent export paths · 8d5f875c
      Stan Hu authored
      When a project export completes, it removes everything in
      `Project#import_export_shared.archive_path`, which can erase files
      needed for another ongoing project export. This is problematic for
      custom templates, which exports an existing project to get the most
      recent changes and imports that archive to another project.
      
      To avoid this from happening, we generate a random unique subpath in the
      shared temporary directory so that multiple exports can work at the same
      time.
      
      Previously the path structure was as follows:
      
      1. Project export files stored in:
         /shared/tmp/project_exports/namespace/project/:random
      2. Project export .tar.gz files stored in:
         /shared/tmp/project_exports/namespace/project
      3. Project export lock file:
         /shared/tmp/project_exports/namespace/project/.after_export_action
      
      Now:
      
      1. Project export files stored in:
         /shared/tmp/project_exports/namespace/project/:randomA/:randomB
      2. Project export .tar.gz files stored in:
         /shared/tmp/project_exports/namespace/project/:randomA
      3. Project export lock files stored in:
         /shared/tmp/project_exports/namespace/project/locks
      
      The .tar.gz files are also now cleaned up in the AfterExportStrategy.
      
      Also, ensure import/export path cleanup always happens.
      
      A failure to update the database or object storage shouldn't
      block us from cleaning up stale directories. This is especially
      important to clear out stale lock file and archive paths.
      
      Closes https://gitlab.com/gitlab-org/gitlab-ee/issues/14716
      8d5f875c
  2. 18 Sep, 2019 39 commits