Merge branch 'docs-artifacts-delete-invalid-references' into 'master'

Docs - Artifacts - delete invalid references

See merge request gitlab-org/gitlab!62592

doc/administration/job_logs.md

@@ -128,6 +128,11 @@ This command permanently deletes the log files and is irreversible.
 find /var/opt/gitlab/gitlab-rails/shared/artifacts -name "job.log" -mtime +60 -delete
 ```
 
+NOTE:
+After execution, broken file references can be reported when running
+[`sudo gitlab-rake gitlab:artifacts:check`](raketasks/check.md#uploaded-files-integrity).
+For more information, see [delete references to missing artifacts](raketasks/check.md#delete-references-to-missing-artifacts).
+
 ## Incremental logging architecture
 
 > - [Deployed behind a feature flag](../user/feature_flags.md), disabled by default.

doc/administration/raketasks/check.md

@@ -246,6 +246,41 @@ end
 p "#{uploads_deleted} remote objects were destroyed."
 ```
 
+### Delete references to missing artifacts
+
+`gitlab-rake gitlab:artifacts:check VERBOSE=1` detects when artifacts (or `job.log` files):
+
+- Are deleted outside of GitLab.
+- Have references still in the GitLab database.
+
+When this scenario is detected, the Rake task displays an error message. For example:
+
+```shell
+Checking integrity of Job artifacts
+- 3..8: Failures: 2
+  - Job artifact: 3: #<Errno::ENOENT: No such file or directory @ rb_sysopen - /var/opt/gitlab/gitlab-rails/shared/artifacts/4e/07/4e07408562bedb8b60ce05c1decfe3ad16b72230967de01f640b7e4729b49fce/2021_05_26/5/3/job.log>
+  - Job artifact: 8: #<Errno::ENOENT: No such file or directory @ rb_sysopen - /var/opt/gitlab/gitlab-rails/shared/artifacts/4e/07/4e07408562bedb8b60ce05c1decfe3ad16b72230967de01f640b7e4729b49fce/2021_05_26/6/8/job.log>
+Done!
+
+```
+
+To delete these references to missing local artifacts (`job.log` files):
+
+1. Open the [GitLab Rails Console](../operations/rails_console.md#starting-a-rails-console-session).
+1. Run the following Ruby code:
+
+   ```ruby
+   artifacts_deleted = 0
+   ::Ci::JobArtifact.all.each do |artifact|                       ### Iterate artifacts
+   #  next if artifact.file.filename != "job.log"                 ### Uncomment if only `job.log` files' references are to be processed
+     next if artifact.file.exists?                                ### Skip if the file reference is valid
+     artifacts_deleted += 1
+     puts "#{artifact.id}  #{artifact.file.path} is missing."     ### Allow verification before destroy
+   #  artifact.destroy!                                           ### Uncomment to actually destroy
+   end
+   puts "Count of identified/destroyed invalid references: #{artifacts_deleted}" 
+   ```
+
 ### Delete references to missing LFS objects
 
 If `gitlab-rake gitlab:lfs:check VERBOSE=1` detects LFS objects that exist in the database