Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
G
gitlab-ce
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
1
Merge Requests
1
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
nexedi
gitlab-ce
Commits
39229eed
Commit
39229eed
authored
Jun 13, 2019
by
Gabriel Mazetto
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Hashed Storage is enabled by default on new installations
updated documentation for Geo
parent
d2929d6e
Changes
9
Show whitespace changes
Inline
Side-by-side
Showing
9 changed files
with
33 additions
and
26 deletions
+33
-26
changelogs/unreleased/hashed-storage-enabled-default.yml
changelogs/unreleased/hashed-storage-enabled-default.yml
+5
-0
db/migrate/20190613030606_enable_hashed_storage_by_default.rb
...igrate/20190613030606_enable_hashed_storage_by_default.rb
+15
-0
db/schema.rb
db/schema.rb
+2
-2
doc/administration/repository_storage_types.md
doc/administration/repository_storage_types.md
+4
-10
spec/finders/fork_projects_finder_spec.rb
spec/finders/fork_projects_finder_spec.rb
+2
-0
spec/models/project_spec.rb
spec/models/project_spec.rb
+1
-12
spec/services/projects/after_rename_service_spec.rb
spec/services/projects/after_rename_service_spec.rb
+1
-0
spec/services/projects/create_service_spec.rb
spec/services/projects/create_service_spec.rb
+1
-1
spec/services/projects/fork_service_spec.rb
spec/services/projects/fork_service_spec.rb
+2
-1
No files found.
changelogs/unreleased/hashed-storage-enabled-default.yml
0 → 100644
View file @
39229eed
---
title
:
Hashed Storage is enabled by default on new installations
merge_request
:
29586
author
:
type
:
changed
db/migrate/20190613030606_enable_hashed_storage_by_default.rb
0 → 100644
View file @
39229eed
# frozen_string_literal: true
class
EnableHashedStorageByDefault
<
ActiveRecord
::
Migration
[
5.1
]
include
Gitlab
::
Database
::
MigrationHelpers
DOWNTIME
=
false
def
up
change_column_default
:application_settings
,
:hashed_storage_enabled
,
true
end
def
down
change_column_default
:application_settings
,
:hashed_storage_enabled
,
false
end
end
db/schema.rb
View file @
39229eed
...
@@ -10,7 +10,7 @@
...
@@ -10,7 +10,7 @@
#
#
# It's strongly recommended that you check this file into your version control system.
# It's strongly recommended that you check this file into your version control system.
ActiveRecord
::
Schema
.
define
(
version:
2019061
1161641
)
do
ActiveRecord
::
Schema
.
define
(
version:
2019061
3030606
)
do
# These are extensions that must be enabled in order to support this database
# These are extensions that must be enabled in order to support this database
enable_extension
"plpgsql"
enable_extension
"plpgsql"
...
@@ -141,7 +141,7 @@ ActiveRecord::Schema.define(version: 20190611161641) do
...
@@ -141,7 +141,7 @@ ActiveRecord::Schema.define(version: 20190611161641) do
t
.
boolean
"help_page_hide_commercial_content"
,
default:
false
t
.
boolean
"help_page_hide_commercial_content"
,
default:
false
t
.
string
"help_page_support_url"
t
.
string
"help_page_support_url"
t
.
integer
"performance_bar_allowed_group_id"
t
.
integer
"performance_bar_allowed_group_id"
t
.
boolean
"hashed_storage_enabled"
,
default:
fals
e
,
null:
false
t
.
boolean
"hashed_storage_enabled"
,
default:
tru
e
,
null:
false
t
.
boolean
"project_export_enabled"
,
default:
true
,
null:
false
t
.
boolean
"project_export_enabled"
,
default:
true
,
null:
false
t
.
boolean
"auto_devops_enabled"
,
default:
true
,
null:
false
t
.
boolean
"auto_devops_enabled"
,
default:
true
,
null:
false
t
.
boolean
"throttle_unauthenticated_enabled"
,
default:
false
,
null:
false
t
.
boolean
"throttle_unauthenticated_enabled"
,
default:
false
,
null:
false
...
...
doc/administration/repository_storage_types.md
View file @
39229eed
...
@@ -47,18 +47,12 @@ Any change in the URL will need to be reflected on disk (when groups / users or
...
@@ -47,18 +47,12 @@ Any change in the URL will need to be reflected on disk (when groups / users or
projects are renamed). This can add a lot of load in big installations,
projects are renamed). This can add a lot of load in big installations,
especially if using any type of network based filesystem.
especially if using any type of network based filesystem.
CAUTION:
**Caution:**
For Geo in particular: Geo does work with legacy storage, but in some
edge cases due to race conditions it can lead to errors when a project is
renamed multiple times in short succession, or a project is deleted and
recreated under the same name very quickly. We expect these race events to be
rare, and we have not observed a race condition side-effect happening yet.
This pattern also exists in other objects stored in GitLab, like issue
Attachments, GitLab Pages artifacts, Docker Containers for the integrated
Registry, etc. Hashed storage is a requirement for Geo.
## Hashed Storage
## Hashed Storage
CAUTION:
**Important:**
Geo requires Hashed Storage since 12.0. If you haven't migrated yet,
check the
[
migration instructions
](
#how-to-migrate-to-hashed-storage
)
ASAP.
Hashed Storage is the new storage behavior we rolled out with 10.0. Instead
Hashed Storage is the new storage behavior we rolled out with 10.0. Instead
of coupling project URL and the folder structure where the repository will be
of coupling project URL and the folder structure where the repository will be
stored on disk, we are coupling a hash, based on the project's ID. This makes
stored on disk, we are coupling a hash, based on the project's ID. This makes
...
...
spec/finders/fork_projects_finder_spec.rb
View file @
39229eed
...
@@ -12,6 +12,8 @@ describe ForkProjectsFinder do
...
@@ -12,6 +12,8 @@ describe ForkProjectsFinder do
let
(
:private_fork_member
)
{
create
(
:user
)
}
let
(
:private_fork_member
)
{
create
(
:user
)
}
before
do
before
do
stub_feature_flags
(
object_pools:
{
enabled:
false
,
thing:
source_project
})
private_fork
.
update!
(
visibility_level:
Gitlab
::
VisibilityLevel
::
PRIVATE
)
private_fork
.
update!
(
visibility_level:
Gitlab
::
VisibilityLevel
::
PRIVATE
)
private_fork
.
add_developer
(
private_fork_member
)
private_fork
.
add_developer
(
private_fork_member
)
...
...
spec/models/project_spec.rb
View file @
39229eed
...
@@ -3478,6 +3478,7 @@ describe Project do
...
@@ -3478,6 +3478,7 @@ describe Project do
before
do
before
do
allow
(
project
).
to
receive
(
:gitlab_shell
).
and_return
(
gitlab_shell
)
allow
(
project
).
to
receive
(
:gitlab_shell
).
and_return
(
gitlab_shell
)
stub_application_setting
(
hashed_storage_enabled:
false
)
end
end
describe
'#base_dir'
do
describe
'#base_dir'
do
...
@@ -3584,10 +3585,6 @@ describe Project do
...
@@ -3584,10 +3585,6 @@ describe Project do
let
(
:hashed_prefix
)
{
File
.
join
(
'@hashed'
,
hash
[
0
..
1
],
hash
[
2
..
3
])
}
let
(
:hashed_prefix
)
{
File
.
join
(
'@hashed'
,
hash
[
0
..
1
],
hash
[
2
..
3
])
}
let
(
:hashed_path
)
{
File
.
join
(
hashed_prefix
,
hash
)
}
let
(
:hashed_path
)
{
File
.
join
(
hashed_prefix
,
hash
)
}
before
do
stub_application_setting
(
hashed_storage_enabled:
true
)
end
describe
'#legacy_storage?'
do
describe
'#legacy_storage?'
do
it
'returns false'
do
it
'returns false'
do
expect
(
project
.
legacy_storage?
).
to
be_falsey
expect
(
project
.
legacy_storage?
).
to
be_falsey
...
@@ -4729,10 +4726,6 @@ describe Project do
...
@@ -4729,10 +4726,6 @@ describe Project do
subject
{
project
.
object_pool_params
}
subject
{
project
.
object_pool_params
}
before
do
stub_application_setting
(
hashed_storage_enabled:
true
)
end
context
'when the objects cannot be pooled'
do
context
'when the objects cannot be pooled'
do
let
(
:project
)
{
create
(
:project
,
:repository
,
:private
)
}
let
(
:project
)
{
create
(
:project
,
:repository
,
:private
)
}
...
@@ -4778,10 +4771,6 @@ describe Project do
...
@@ -4778,10 +4771,6 @@ describe Project do
context
'when objects are poolable'
do
context
'when objects are poolable'
do
let
(
:project
)
{
create
(
:project
,
:repository
,
:public
)
}
let
(
:project
)
{
create
(
:project
,
:repository
,
:public
)
}
before
do
stub_application_setting
(
hashed_storage_enabled:
true
)
end
it
{
is_expected
.
to
be_git_objects_poolable
}
it
{
is_expected
.
to
be_git_objects_poolable
}
end
end
end
end
...
...
spec/services/projects/after_rename_service_spec.rb
View file @
39229eed
...
@@ -23,6 +23,7 @@ describe Projects::AfterRenameService do
...
@@ -23,6 +23,7 @@ describe Projects::AfterRenameService do
allow
(
project
).
to
receive
(
:gitlab_shell
).
and_return
(
gitlab_shell
)
allow
(
project
).
to
receive
(
:gitlab_shell
).
and_return
(
gitlab_shell
)
stub_feature_flags
(
skip_hashed_storage_upgrade:
false
)
stub_feature_flags
(
skip_hashed_storage_upgrade:
false
)
stub_application_setting
(
hashed_storage_enabled:
false
)
end
end
it
'renames a repository'
do
it
'renames a repository'
do
...
...
spec/services/projects/create_service_spec.rb
View file @
39229eed
...
@@ -228,6 +228,7 @@ describe Projects::CreateService, '#execute' do
...
@@ -228,6 +228,7 @@ describe Projects::CreateService, '#execute' do
context
'with legacy storage'
do
context
'with legacy storage'
do
before
do
before
do
stub_application_setting
(
hashed_storage_enabled:
false
)
gitlab_shell
.
create_repository
(
repository_storage
,
"
#{
user
.
namespace
.
full_path
}
/existing"
,
'group/project'
)
gitlab_shell
.
create_repository
(
repository_storage
,
"
#{
user
.
namespace
.
full_path
}
/existing"
,
'group/project'
)
end
end
...
@@ -259,7 +260,6 @@ describe Projects::CreateService, '#execute' do
...
@@ -259,7 +260,6 @@ describe Projects::CreateService, '#execute' do
let
(
:hashed_path
)
{
'@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b'
}
let
(
:hashed_path
)
{
'@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b'
}
before
do
before
do
stub_application_setting
(
hashed_storage_enabled:
true
)
allow
(
Digest
::
SHA2
).
to
receive
(
:hexdigest
)
{
hash
}
allow
(
Digest
::
SHA2
).
to
receive
(
:hexdigest
)
{
hash
}
end
end
...
...
spec/services/projects/fork_service_spec.rb
View file @
39229eed
...
@@ -116,11 +116,12 @@ describe Projects::ForkService do
...
@@ -116,11 +116,12 @@ describe Projects::ForkService do
end
end
end
end
context
'repository already exists'
do
context
'repository
in legacy storage
already exists'
do
let
(
:repository_storage
)
{
'default'
}
let
(
:repository_storage
)
{
'default'
}
let
(
:repository_storage_path
)
{
Gitlab
.
config
.
repositories
.
storages
[
repository_storage
].
legacy_disk_path
}
let
(
:repository_storage_path
)
{
Gitlab
.
config
.
repositories
.
storages
[
repository_storage
].
legacy_disk_path
}
before
do
before
do
stub_application_setting
(
hashed_storage_enabled:
false
)
gitlab_shell
.
create_repository
(
repository_storage
,
"
#{
@to_user
.
namespace
.
full_path
}
/
#{
@from_project
.
path
}
"
,
"
#{
@to_user
.
namespace
.
full_path
}
/
#{
@from_project
.
path
}
"
)
gitlab_shell
.
create_repository
(
repository_storage
,
"
#{
@to_user
.
namespace
.
full_path
}
/
#{
@from_project
.
path
}
"
,
"
#{
@to_user
.
namespace
.
full_path
}
/
#{
@from_project
.
path
}
"
)
end
end
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment