Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
G
gitlab-ce
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
Léo-Paul Géneau
gitlab-ce
Commits
f3a93151
Commit
f3a93151
authored
Oct 28, 2018
by
Andreas Brandl
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Extract ReltuplesCountStrategy.
parent
c3c25174
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
38 additions
and
21 deletions
+38
-21
lib/gitlab/database/count.rb
lib/gitlab/database/count.rb
+37
-20
spec/lib/gitlab/database/count_spec.rb
spec/lib/gitlab/database/count_spec.rb
+1
-1
No files found.
lib/gitlab/database/count.rb
View file @
f3a93151
...
@@ -54,34 +54,51 @@ module Gitlab
...
@@ -54,34 +54,51 @@ module Gitlab
# @param [Array] table names
# @param [Array] table names
# @returns [Hash] Table name to count mapping (e.g. { 'projects' => 5, 'users' => 100 })
# @returns [Hash] Table name to count mapping (e.g. { 'projects' => 5, 'users' => 100 })
def
self
.
reltuples_from_recently_updated
(
table_names
)
def
self
.
reltuples_from_recently_updated
(
table_names
)
query
=
postgresql_estimate_query
(
table_names
)
ReltuplesCountStrategy
.
new
(
table_names
).
count
rows
=
[]
end
class
ReltuplesCountStrategy
attr_reader
:table_names
# Querying tuple stats only works on the primary. Due to load
# @param [Array] table names
# balancing, we need to ensure this query hits the load balancer. The
def
initialize
(
table_names
)
# easiest way to do this is to start a transaction.
@table_names
=
table_names
ActiveRecord
::
Base
.
transaction
do
rows
=
ActiveRecord
::
Base
.
connection
.
select_all
(
query
)
end
end
rows
.
each_with_object
({})
{
|
row
,
data
|
data
[
row
[
'table_name'
]]
=
row
[
'estimate'
].
to_i
}
# Returns a hash of the table names that have recently updated tuples.
rescue
*
CONNECTION_ERRORS
#
{}
# @returns [Hash] Table name to count mapping (e.g. { 'projects' => 5, 'users' => 100 })
end
def
count
query
=
postgresql_estimate_query
(
table_names
)
rows
=
[]
# Generates the PostgreSQL query to return the tuples for tables
# Querying tuple stats only works on the primary. Due to load
# that have been vacuumed or analyzed in the last hour.
# easiest way to do this is to start a transaction.
#
ActiveRecord
::
Base
.
transaction
do
# @param [Array] table names
rows
=
ActiveRecord
::
Base
.
connection
.
select_all
(
query
)
# @returns [Hash] Table name to count mapping (e.g. { 'projects' => 5, 'users' => 100 })
end
def
self
.
postgresql_estimate_query
(
table_names
)
time
=
"to_timestamp(
#{
1
.
hour
.
ago
.
to_i
}
)"
rows
.
each_with_object
({})
{
|
row
,
data
|
data
[
row
[
'table_name'
]]
=
row
[
'estimate'
].
to_i
}
<<~
SQL
rescue
*
CONNECTION_ERRORS
=>
e
{}
end
private
# Generates the PostgreSQL query to return the tuples for tables
# that have been vacuumed or analyzed in the last hour.
#
# @param [Array] table names
# @returns [Hash] Table name to count mapping (e.g. { 'projects' => 5, 'users' => 100 })
def
postgresql_estimate_query
(
table_names
)
time
=
"to_timestamp(
#{
1
.
hour
.
ago
.
to_i
}
)"
<<~
SQL
SELECT pg_class.relname AS table_name, reltuples::bigint AS estimate FROM pg_class
SELECT pg_class.relname AS table_name, reltuples::bigint AS estimate FROM pg_class
LEFT JOIN pg_stat_user_tables ON pg_class.relname = pg_stat_user_tables.relname
LEFT JOIN pg_stat_user_tables ON pg_class.relname = pg_stat_user_tables.relname
WHERE pg_class.relname IN (
#{
table_names
.
map
{
|
table
|
"'
#{
table
}
'"
}
.join(',')})
WHERE pg_class.relname IN (
#{
table_names
.
map
{
|
table
|
"'
#{
table
}
'"
}
.join(',')})
AND (last_vacuum >
#{
time
}
OR last_autovacuum >
#{
time
}
OR last_analyze >
#{
time
}
OR last_autoanalyze >
#{
time
}
)
AND (last_vacuum >
#{
time
}
OR last_autovacuum >
#{
time
}
OR last_analyze >
#{
time
}
OR last_autoanalyze >
#{
time
}
)
SQL
SQL
end
end
end
end
end
end
end
...
...
spec/lib/gitlab/database/count_spec.rb
View file @
f3a93151
...
@@ -35,7 +35,7 @@ describe Gitlab::Database::Count do
...
@@ -35,7 +35,7 @@ describe Gitlab::Database::Count do
describe
'no permission'
do
describe
'no permission'
do
it
'falls back to standard query'
do
it
'falls back to standard query'
do
allow
(
described_class
).
to
receive
(
:postgresql_estimate_query
).
and_raise
(
PG
::
InsufficientPrivilege
)
allow
(
ActiveRecord
::
Base
).
to
receive
(
:transaction
).
and_raise
(
PG
::
InsufficientPrivilege
)
expect
(
Project
).
to
receive
(
:count
).
and_call_original
expect
(
Project
).
to
receive
(
:count
).
and_call_original
expect
(
Identity
).
to
receive
(
:count
).
and_call_original
expect
(
Identity
).
to
receive
(
:count
).
and_call_original
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment