Commit 9a66404e authored by Amy Qualls's avatar Amy Qualls

Merge branch 'bbodenmiller-master-patch-00271' into 'master'

Move admin facing LFS details to LFS admin page & improve

See merge request gitlab-org/gitlab!64867
parents 8a9bf111 238316a2
...@@ -243,7 +243,52 @@ You can see the total storage used for LFS objects on groups and projects: ...@@ -243,7 +243,52 @@ You can see the total storage used for LFS objects on groups and projects:
- In the administration area. - In the administration area.
- In the [groups](../../api/groups.md) and [projects APIs](../../api/projects.md). - In the [groups](../../api/groups.md) and [projects APIs](../../api/projects.md).
## Troubleshooting: `Google::Apis::TransmissionError: execution expired` ## Troubleshooting
### Missing LFS objects
An error about a missing LFS object may occur in either of these situations:
- When migrating LFS objects from disk to object storage, with error messages like:
```plaintext
ERROR -- : Failed to transfer LFS object
006622269c61b41bf14a22bbe0e43be3acf86a4a446afb4250c3794ea47541a7
with error: No such file or directory @ rb_sysopen -
/var/opt/gitlab/gitlab-rails/shared/lfs-objects/00/66/22269c61b41bf14a22bbe0e43be3acf86a4a446afb4250c3794ea47541a7
```
(Line breaks have been added for legibility.)
- When running the
[integrity check for LFS objects](../raketasks/check.md#uploaded-files-integrity)
with the `VERBOSE=1` parameter.
The database can have records for LFS objects which are not on disk. The database entry may
[prevent a new copy of the object from being pushed](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/49241).
To delete these references:
1. [Start a rails console](../operations/rails_console.md).
1. Query the object that's reported as missing in the rails console, to return a file path:
```ruby
lfs_object = LfsObject.find_by(oid: '006622269c61b41bf14a22bbe0e43be3acf86a4a446afb4250c3794ea47541a7')
lfs_object.file.path
```
1. Check on disk or object storage if it exists:
```shell
ls -al /var/opt/gitlab/gitlab-rails/shared/lfs-objects/00/66/22269c61b41bf14a22bbe0e43be3acf86a4a446afb4250c3794ea47541a7
```
1. If the file is not present, remove the database record via the rails console:
```ruby
lfs_object.destroy
```
### `Google::Apis::TransmissionError: execution expired`
If LFS integration is configured with Google Cloud Storage and background uploads (`background_upload: true` and `direct_upload: false`), If LFS integration is configured with Google Cloud Storage and background uploads (`background_upload: true` and `direct_upload: false`),
Sidekiq workers may encounter this error. This is because the uploading timed out with very large files. Sidekiq workers may encounter this error. This is because the uploading timed out with very large files.
...@@ -281,5 +326,5 @@ See more information in [!19581](https://gitlab.com/gitlab-org/gitlab-foss/-/mer ...@@ -281,5 +326,5 @@ See more information in [!19581](https://gitlab.com/gitlab-org/gitlab-foss/-/mer
- Support for removing unreferenced LFS objects was added in 8.14 onward. - Support for removing unreferenced LFS objects was added in 8.14 onward.
- LFS authentications via SSH was added with GitLab 8.12. - LFS authentications via SSH was added with GitLab 8.12.
- Only compatible with the Git LFS client versions 1.1.0 and later, or 1.0.2. - Only compatible with the Git LFS client versions 1.1.0 and later, or 1.0.2.
- The storage statistics count each LFS object multiple times for - The storage statistics count each LFS object for
every project linking to it. every project linking to it.
...@@ -278,11 +278,11 @@ To delete these references to missing local artifacts (`job.log` files): ...@@ -278,11 +278,11 @@ To delete these references to missing local artifacts (`job.log` files):
puts "#{artifact.id} #{artifact.file.path} is missing." ### Allow verification before destroy puts "#{artifact.id} #{artifact.file.path} is missing." ### Allow verification before destroy
# artifact.destroy! ### Uncomment to actually destroy # artifact.destroy! ### Uncomment to actually destroy
end end
puts "Count of identified/destroyed invalid references: #{artifacts_deleted}" puts "Count of identified/destroyed invalid references: #{artifacts_deleted}"
``` ```
### Delete references to missing LFS objects ### Delete references to missing LFS objects
If `gitlab-rake gitlab:lfs:check VERBOSE=1` detects LFS objects that exist in the database If `gitlab-rake gitlab:lfs:check VERBOSE=1` detects LFS objects that exist in the database
but not on disk, [follow the procedure in the LFS documentation](../../topics/git/lfs/index.md#missing-lfs-objects) but not on disk, [follow the procedure in the LFS documentation](../lfs/index.md#missing-lfs-objects)
to remove the database entries. to remove the database entries.
...@@ -275,46 +275,3 @@ You might choose to do this if you are using an appliance like a <!-- vale gitla ...@@ -275,46 +275,3 @@ You might choose to do this if you are using an appliance like a <!-- vale gitla
GitLab can't verify LFS objects. Pushes then fail if you have GitLab LFS support enabled. GitLab can't verify LFS objects. Pushes then fail if you have GitLab LFS support enabled.
To stop push failure, LFS support can be disabled in the [Project settings](../../../user/project/settings/index.md), which also disables GitLab LFS value-adds (Verifying LFS objects, UI integration for LFS). To stop push failure, LFS support can be disabled in the [Project settings](../../../user/project/settings/index.md), which also disables GitLab LFS value-adds (Verifying LFS objects, UI integration for LFS).
### Missing LFS objects
An error about a missing LFS object may occur in either of these situations:
- When migrating LFS objects from disk to object storage, with error messages like:
```plaintext
ERROR -- : Failed to transfer LFS object
006622269c61b41bf14a22bbe0e43be3acf86a4a446afb4250c3794ea47541a7
with error: No such file or directory @ rb_sysopen -
/var/opt/gitlab/gitlab-rails/shared/lfs-objects/00/66/22269c61b41bf14a22bbe0e43be3acf86a4a446afb4250c3794ea47541a7
```
(Line breaks have been added for legibility.)
- When running the
[integrity check for LFS objects](../../../administration/raketasks/check.md#uploaded-files-integrity)
with the `VERBOSE=1` parameter.
The database can have records for LFS objects which are not on disk. The database entry may
[prevent a new copy of the object being pushed](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/49241).
To delete these references:
1. [Start a rails console](../../../administration/operations/rails_console.md).
1. Query the object that's reported as missing in the rails console, to return a file path:
```ruby
lfs_object = LfsObject.find_by(oid: '006622269c61b41bf14a22bbe0e43be3acf86a4a446afb4250c3794ea47541a7')
lfs_object.file.path
```
1. Check on disk if it exists:
```shell
ls -al /var/opt/gitlab/gitlab-rails/shared/lfs-objects/00/66/22269c61b41bf14a22bbe0e43be3acf86a4a446afb4250c3794ea47541a7
```
1. If the file is not present, remove the database record via the rails console:
```ruby
lfs_object.destroy
```
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment