• Kassio Borges's avatar
    Github Importer: Ensure to fail and log imports on exceptions · 752512e0
    Kassio Borges authored
    Currently, the Github Importer might end up stuck even after failing to
    import the git repository. This might also happen when importing other
    types of objects from Github.
    
    To provide better feedback to the end user this kind of exception will
    now mark the import (import_state) as failed. Also, to provide
    information to developers and testers the exceptions errors will be
    logged (importer.log) and saved in the import_failures table.
    
    == What was done
    
    * Created Gitlab::Import::ImportFailureService: This way we now have a
      standard way to log exceptions/errors that happens while importing a
      project;
      * All the exceptions/errors are logged to the importer.log with a
        standard format;
      * All the exceptions/errors are tracked in Gitlab::ErrorTracking;
      * All the exceptions/errors are recorded in the import_failures table;
      * When required, the exception/error might mark the whole import as
        failed;
      * This was created in a more generic way because it can later be used
        in the other importers (bitbucket, gitlab, etc);
    
    * Updated Github Importer workers/importers to leverage of the
      Gitlab::Import::ImportFailureService;
    
    * Updated Gitlab::GithubImport::Queue, to use
      Gitlab::Import::ImportFailureService when jobs exhausted retries;
    
    * Updated Gitlab::Import::StuckImportJob to use
      Gitlab::Import::ImportFailureService;
      * This is also used in the Jira importer;
    
    Related to: https://gitlab.com/gitlab-org/gitlab/-/issues/335660
    
    Changelog: fixed
    MR: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/67454
    752512e0
parallel_scheduling_spec.rb 11.3 KB