Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
G
gitlab-ce
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
Boxiang Sun
gitlab-ce
Commits
3082b7d1
Commit
3082b7d1
authored
Jul 09, 2018
by
Jacob Vosmaer (GitLab)
Committed by
Douwe Maan
Jul 09, 2018
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Make blob and other RPC's mandatory
parent
090f4d9e
Changes
5
Hide whitespace changes
Inline
Side-by-side
Showing
5 changed files
with
36 additions
and
171 deletions
+36
-171
lib/gitlab/git/blob.rb
lib/gitlab/git/blob.rb
+2
-17
lib/gitlab/git/repository.rb
lib/gitlab/git/repository.rb
+8
-108
lib/gitlab/workhorse.rb
lib/gitlab/workhorse.rb
+8
-15
spec/lib/gitlab/git/blob_spec.rb
spec/lib/gitlab/git/blob_spec.rb
+3
-4
spec/lib/gitlab/workhorse_spec.rb
spec/lib/gitlab/workhorse_spec.rb
+15
-27
No files found.
lib/gitlab/git/blob.rb
View file @
3082b7d1
...
@@ -50,13 +50,7 @@ module Gitlab
...
@@ -50,13 +50,7 @@ module Gitlab
end
end
def
raw
(
repository
,
sha
)
def
raw
(
repository
,
sha
)
Gitlab
::
GitalyClient
.
migrate
(
:git_blob_raw
)
do
|
is_enabled
|
repository
.
gitaly_blob_client
.
get_blob
(
oid:
sha
,
limit:
MAX_DATA_DISPLAY_SIZE
)
if
is_enabled
repository
.
gitaly_blob_client
.
get_blob
(
oid:
sha
,
limit:
MAX_DATA_DISPLAY_SIZE
)
else
rugged_raw
(
repository
,
sha
,
limit:
MAX_DATA_DISPLAY_SIZE
)
end
end
end
end
# Returns an array of Blob instances, specified in blob_references as
# Returns an array of Blob instances, specified in blob_references as
...
@@ -207,16 +201,7 @@ module Gitlab
...
@@ -207,16 +201,7 @@ module Gitlab
return
if
@loaded_all_data
return
if
@loaded_all_data
@data
=
Gitlab
::
GitalyClient
.
migrate
(
:git_blob_load_all_data
)
do
|
is_enabled
|
@data
=
repository
.
gitaly_blob_client
.
get_blob
(
oid:
id
,
limit:
-
1
).
data
begin
if
is_enabled
repository
.
gitaly_blob_client
.
get_blob
(
oid:
id
,
limit:
-
1
).
data
else
repository
.
lookup
(
id
).
content
end
end
end
@loaded_all_data
=
true
@loaded_all_data
=
true
@loaded_size
=
@data
.
bytesize
@loaded_size
=
@data
.
bytesize
end
end
...
...
lib/gitlab/git/repository.rb
View file @
3082b7d1
...
@@ -439,31 +439,11 @@ module Gitlab
...
@@ -439,31 +439,11 @@ module Gitlab
raise
ArgumentError
.
new
(
"invalid Repository#log limit:
#{
limit
.
inspect
}
"
)
raise
ArgumentError
.
new
(
"invalid Repository#log limit:
#{
limit
.
inspect
}
"
)
end
end
gitaly_migrate
(
:find_commits
)
do
|
is_enabled
|
wrapped_gitaly_errors
do
if
is_enabled
gitaly_commit_client
.
find_commits
(
options
)
gitaly_commit_client
.
find_commits
(
options
)
else
raw_log
(
options
).
map
{
|
c
|
Commit
.
decorate
(
self
,
c
)
}
end
end
end
end
end
# Used in gitaly-ruby
def
raw_log
(
options
)
sha
=
unless
options
[
:all
]
actual_ref
=
options
[
:ref
]
||
root_ref
begin
sha_from_ref
(
actual_ref
)
rescue
Rugged
::
OdbError
,
Rugged
::
InvalidError
,
Rugged
::
ReferenceError
# Return an empty array if the ref wasn't found
return
[]
end
end
log_by_shell
(
sha
,
options
)
end
def
count_commits
(
options
)
def
count_commits
(
options
)
options
=
process_count_commits_options
(
options
.
dup
)
options
=
process_count_commits_options
(
options
.
dup
)
...
@@ -1223,12 +1203,10 @@ module Gitlab
...
@@ -1223,12 +1203,10 @@ module Gitlab
end
end
def
find_commits_by_message
(
query
,
ref
,
path
,
limit
,
offset
)
def
find_commits_by_message
(
query
,
ref
,
path
,
limit
,
offset
)
gitaly_migrate
(
:commits_by_message
)
do
|
is_enabled
|
wrapped_gitaly_errors
do
if
is_enabled
gitaly_commit_client
find_commits_by_message_by_gitaly
(
query
,
ref
,
path
,
limit
,
offset
)
.
commits_by_message
(
query
,
revision:
ref
,
path:
path
,
limit:
limit
,
offset:
offset
)
else
.
map
{
|
c
|
commit
(
c
)
}
find_commits_by_message_by_shelling_out
(
query
,
ref
,
path
,
limit
,
offset
)
end
end
end
end
end
...
@@ -1238,12 +1216,8 @@ module Gitlab
...
@@ -1238,12 +1216,8 @@ module Gitlab
end
end
def
last_commit_for_path
(
sha
,
path
)
def
last_commit_for_path
(
sha
,
path
)
gitaly_migrate
(
:last_commit_for_path
)
do
|
is_enabled
|
wrapped_gitaly_errors
do
if
is_enabled
gitaly_commit_client
.
last_commit_for_path
(
sha
,
path
)
last_commit_for_path_by_gitaly
(
sha
,
path
)
else
last_commit_for_path_by_rugged
(
sha
,
path
)
end
end
end
end
end
...
@@ -1454,46 +1428,6 @@ module Gitlab
...
@@ -1454,46 +1428,6 @@ module Gitlab
end
end
end
end
# Gitaly note: JV: although #log_by_shell shells out to Git I think the
# complexity is such that we should migrate it as Ruby before trying to
# do it in Go.
def
log_by_shell
(
sha
,
options
)
limit
=
options
[
:limit
].
to_i
offset
=
options
[
:offset
].
to_i
use_follow_flag
=
options
[
:follow
]
&&
options
[
:path
].
present?
# We will perform the offset in Ruby because --follow doesn't play well with --skip.
# See: https://gitlab.com/gitlab-org/gitlab-ce/issues/3574#note_3040520
offset_in_ruby
=
use_follow_flag
&&
options
[
:offset
].
present?
limit
+=
offset
if
offset_in_ruby
cmd
=
%w[log]
cmd
<<
"--max-count=
#{
limit
}
"
cmd
<<
'--format=%H'
cmd
<<
"--skip=
#{
offset
}
"
unless
offset_in_ruby
cmd
<<
'--follow'
if
use_follow_flag
cmd
<<
'--no-merges'
if
options
[
:skip_merges
]
cmd
<<
"--after=
#{
options
[
:after
].
iso8601
}
"
if
options
[
:after
]
cmd
<<
"--before=
#{
options
[
:before
].
iso8601
}
"
if
options
[
:before
]
if
options
[
:all
]
cmd
+=
%w[--all --reverse]
else
cmd
<<
sha
end
# :path can be a string or an array of strings
if
options
[
:path
].
present?
cmd
<<
'--'
cmd
+=
Array
(
options
[
:path
])
end
raw_output
,
_status
=
run_git
(
cmd
)
lines
=
offset_in_ruby
?
raw_output
.
lines
.
drop
(
offset
)
:
raw_output
.
lines
lines
.
map!
{
|
c
|
Rugged
::
Commit
.
new
(
rugged
,
c
.
strip
)
}
end
# We are trying to deprecate this method because it does a lot of work
# We are trying to deprecate this method because it does a lot of work
# but it seems to be used only to look up submodule URL's.
# but it seems to be used only to look up submodule URL's.
# https://gitlab.com/gitlab-org/gitaly/issues/329
# https://gitlab.com/gitlab-org/gitaly/issues/329
...
@@ -1623,11 +1557,6 @@ module Gitlab
...
@@ -1623,11 +1557,6 @@ module Gitlab
end
end
end
end
def
last_commit_for_path_by_rugged
(
sha
,
path
)
sha
=
last_commit_id_for_path_by_shelling_out
(
sha
,
path
)
commit
(
sha
)
end
# Returns true if the given ref name exists
# Returns true if the given ref name exists
#
#
# Ref names must start with `refs/`.
# Ref names must start with `refs/`.
...
@@ -1820,35 +1749,6 @@ module Gitlab
...
@@ -1820,35 +1749,6 @@ module Gitlab
raise
CommandError
,
@gitlab_projects
.
output
raise
CommandError
,
@gitlab_projects
.
output
end
end
def
find_commits_by_message_by_shelling_out
(
query
,
ref
,
path
,
limit
,
offset
)
ref
||=
root_ref
args
=
%W(
log
#{
ref
}
--pretty=%H --skip
#{
offset
}
--max-count
#{
limit
}
--grep=
#{
query
}
--regexp-ignore-case
)
args
=
args
.
concat
(
%W(--
#{
path
}
)
)
if
path
.
present?
git_log_results
=
run_git
(
args
).
first
.
lines
git_log_results
.
map
{
|
c
|
commit
(
c
.
chomp
)
}.
compact
end
def
find_commits_by_message_by_gitaly
(
query
,
ref
,
path
,
limit
,
offset
)
gitaly_commit_client
.
commits_by_message
(
query
,
revision:
ref
,
path:
path
,
limit:
limit
,
offset:
offset
)
.
map
{
|
c
|
commit
(
c
)
}
end
def
last_commit_for_path_by_gitaly
(
sha
,
path
)
gitaly_commit_client
.
last_commit_for_path
(
sha
,
path
)
end
def
last_commit_id_for_path_by_shelling_out
(
sha
,
path
)
args
=
%W(rev-list --max-count=1
#{
sha
}
--
#{
path
}
)
run_git_with_timeout
(
args
,
Gitlab
::
Git
::
Popen
::
FAST_GIT_PROCESS_TIMEOUT
).
first
.
strip
end
def
rugged_merge_base
(
from
,
to
)
def
rugged_merge_base
(
from
,
to
)
rugged
.
merge_base
(
from
,
to
)
rugged
.
merge_base
(
from
,
to
)
rescue
Rugged
::
ReferenceError
rescue
Rugged
::
ReferenceError
...
...
lib/gitlab/workhorse.rb
View file @
3082b7d1
...
@@ -37,21 +37,14 @@ module Gitlab
...
@@ -37,21 +37,14 @@ module Gitlab
end
end
def
send_git_blob
(
repository
,
blob
)
def
send_git_blob
(
repository
,
blob
)
params
=
if
Gitlab
::
GitalyClient
.
feature_enabled?
(
:workhorse_raw_show
,
status:
Gitlab
::
GitalyClient
::
MigrationStatus
::
OPT_OUT
)
params
=
{
{
'GitalyServer'
=>
gitaly_server_hash
(
repository
),
'GitalyServer'
=>
gitaly_server_hash
(
repository
),
'GetBlobRequest'
=>
{
'GetBlobRequest'
=>
{
repository:
repository
.
gitaly_repository
.
to_h
,
repository:
repository
.
gitaly_repository
.
to_h
,
oid:
blob
.
id
,
oid:
blob
.
id
,
limit:
-
1
limit:
-
1
}
}
}
}
else
{
'RepoPath'
=>
repository
.
path_to_repo
,
'BlobId'
=>
blob
.
id
}
end
[
[
SEND_DATA_HEADER
,
SEND_DATA_HEADER
,
...
...
spec/lib/gitlab/git/blob_spec.rb
View file @
3082b7d1
...
@@ -532,8 +532,8 @@ describe Gitlab::Git::Blob, seed_helper: true do
...
@@ -532,8 +532,8 @@ describe Gitlab::Git::Blob, seed_helper: true do
subject
{
blob
.
load_all_data!
(
repository
)
}
subject
{
blob
.
load_all_data!
(
repository
)
}
it
'loads missing data'
do
it
'loads missing data'
do
expect
(
Gitlab
::
GitalyClient
).
to
receive
(
:migrate
)
expect
(
repository
.
gitaly_blob_client
).
to
receive
(
:get_blob
)
.
with
(
:git_blob_load_all_data
).
and_return
(
full_data
)
.
and_return
(
double
(
:response
,
data:
full_data
)
)
subject
subject
...
@@ -544,8 +544,7 @@ describe Gitlab::Git::Blob, seed_helper: true do
...
@@ -544,8 +544,7 @@ describe Gitlab::Git::Blob, seed_helper: true do
let
(
:blob
)
{
Gitlab
::
Git
::
Blob
.
new
(
name:
'test'
,
size:
4
,
data:
full_data
)
}
let
(
:blob
)
{
Gitlab
::
Git
::
Blob
.
new
(
name:
'test'
,
size:
4
,
data:
full_data
)
}
it
"doesn't perform any loading"
do
it
"doesn't perform any loading"
do
expect
(
Gitlab
::
GitalyClient
).
not_to
receive
(
:migrate
)
expect
(
repository
.
gitaly_blob_client
).
not_to
receive
(
:get_blob
)
.
with
(
:git_blob_load_all_data
)
subject
subject
...
...
spec/lib/gitlab/workhorse_spec.rb
View file @
3082b7d1
...
@@ -413,34 +413,22 @@ describe Gitlab::Workhorse do
...
@@ -413,34 +413,22 @@ describe Gitlab::Workhorse do
subject
{
described_class
.
send_git_blob
(
repository
,
blob
)
}
subject
{
described_class
.
send_git_blob
(
repository
,
blob
)
}
context
'when Gitaly workhorse_raw_show feature is enabled'
do
it
'sets the header correctly'
do
it
'sets the header correctly'
do
key
,
command
,
params
=
decode_workhorse_header
(
subject
)
key
,
command
,
params
=
decode_workhorse_header
(
subject
)
expect
(
key
).
to
eq
(
'Gitlab-Workhorse-Send-Data'
)
expect
(
command
).
to
eq
(
'git-blob'
)
expect
(
params
).
to
eq
({
'GitalyServer'
=>
{
address:
Gitlab
::
GitalyClient
.
address
(
project
.
repository_storage
),
token:
Gitlab
::
GitalyClient
.
token
(
project
.
repository_storage
)
},
'GetBlobRequest'
=>
{
repository:
repository
.
gitaly_repository
.
to_h
,
oid:
blob
.
id
,
limit:
-
1
}
}.
deep_stringify_keys
)
end
end
context
'when Gitaly workhorse_raw_show feature is disabled'
,
:disable_gitaly
do
it
'sets the header correctly'
do
key
,
command
,
params
=
decode_workhorse_header
(
subject
)
expect
(
key
).
to
eq
(
'Gitlab-Workhorse-Send-Data'
)
expect
(
key
).
to
eq
(
'Gitlab-Workhorse-Send-Data'
)
expect
(
command
).
to
eq
(
'git-blob'
)
expect
(
command
).
to
eq
(
'git-blob'
)
expect
(
params
).
to
eq
(
'RepoPath'
=>
repository
.
path_to_repo
,
'BlobId'
=>
blob
.
id
)
expect
(
params
).
to
eq
({
end
'GitalyServer'
=>
{
address:
Gitlab
::
GitalyClient
.
address
(
project
.
repository_storage
),
token:
Gitlab
::
GitalyClient
.
token
(
project
.
repository_storage
)
},
'GetBlobRequest'
=>
{
repository:
repository
.
gitaly_repository
.
to_h
,
oid:
blob
.
id
,
limit:
-
1
}
}.
deep_stringify_keys
)
end
end
end
end
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment