Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
G
gitlab-ce
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
Boxiang Sun
gitlab-ce
Commits
606584c1
Commit
606584c1
authored
May 05, 2017
by
James Lopez
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
bulk insert FTW - This would introduce more complexity, but should be faster
parent
9ec39568
Changes
2
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
41 additions
and
7 deletions
+41
-7
app/services/projects/propagate_service.rb
app/services/projects/propagate_service.rb
+20
-7
lib/gitlab/sql/bulk_insert.rb
lib/gitlab/sql/bulk_insert.rb
+21
-0
No files found.
app/services/projects/propagate_service.rb
View file @
606584c1
...
@@ -24,20 +24,23 @@ module Projects
...
@@ -24,20 +24,23 @@ module Projects
loop
do
loop
do
batch
=
project_ids_batch
batch
=
project_ids_batch
bulk_create_from_template
(
batch
)
bulk_create_from_template
(
batch
)
unless
batch
.
empty?
break
if
batch
.
size
<
BATCH_SIZE
break
if
batch
.
size
<
BATCH_SIZE
end
end
end
end
def
bulk_create_from_template
(
batch
)
def
bulk_create_from_template
(
batch
)
service_
hash_
list
=
batch
.
map
do
|
project_id
|
service_list
=
batch
.
map
do
|
project_id
|
service_hash
.
merge
(
'project_id'
=>
project_id
)
service_hash
.
merge
(
'project_id'
=>
project_id
)
.
values
end
end
Project
.
transaction
do
# Project.transaction do
Service
.
create!
(
service_hash_list
)
# Service.create!(service_hash_list)
end
# end
Gitlab
::
SQL
::
BulkInsert
.
new
(
service_hash
.
keys
+
[
'project_id'
],
service_list
,
'services'
).
execute
end
end
def
project_ids_batch
def
project_ids_batch
...
@@ -57,7 +60,17 @@ module Projects
...
@@ -57,7 +60,17 @@ module Projects
end
end
def
service_hash
def
service_hash
@service_hash
||=
@template
.
as_json
(
methods: :type
).
except
(
'id'
,
'template'
)
@service_hash
||=
begin
template_hash
=
@template
.
as_json
(
methods: :type
).
except
(
'id'
,
'template'
,
'project_id'
)
template_hash
.
each_with_object
({})
do
|
(
key
,
value
),
service_hash
|
value
=
value
.
is_a?
(
Hash
)
?
value
.
to_json
:
value
key
=
Gitlab
::
Database
.
postgresql?
?
"
\"
#{
key
}
\"
"
:
"`
#{
key
}
`"
service_hash
[
key
]
=
ActiveRecord
::
Base
.
sanitize
(
value
)
end
end
end
end
end
end
end
end
lib/gitlab/sql/bulk_insert.rb
0 → 100644
View file @
606584c1
module
Gitlab
module
SQL
# Class for building SQL bulk inserts
class
BulkInsert
def
initialize
(
columns
,
values_array
,
table
)
@columns
=
columns
@values_array
=
values_array
@table
=
table
end
def
execute
ActiveRecord
::
Base
.
connection
.
execute
(
<<-
SQL
.
strip_heredoc
INSERT INTO
#{
@table
}
(
#{
@columns
.
join
(
', '
)
}
)
VALUES
#{
@values_array
.
map
{
|
tuple
|
"(
#{
tuple
.
join
(
', '
)
}
)"
}
.join(', ')}
SQL
)
end
end
end
end
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment