Commit 389e6f58 authored by Andrew Newdigate's avatar Andrew Newdigate Committed by Stan Hu

Improves the performance of the Redis set cache

Testing has shown that treating membership optimistically reduces the
number of Redis calls substantially.

`include?` will only perform an EXISTS if SISMEMBER returns negative.

`fetch` will only perform an EXISTS if there are no values.

`read` uses SSCAN to avoid Redis lockup on huge sets
parent 13113802
......@@ -58,11 +58,16 @@ module Gitlab
# wrong answer. We handle that by querying the full list - which fills
# the cache - and using it directly to answer the question.
define_method("#{name}_include?") do |value|
if strong_memoized?(name) || !redis_set_cache.exist?(name)
return __send__(name).include?(value) # rubocop:disable GitlabSecurity/PublicSend
end
return __send__(name).include?(value) if strong_memoized?(name) # rubocop:disable GitlabSecurity/PublicSend
# If the member exists in the set, return as such early.
return true if redis_set_cache.include?(name, value)
# If it did not, make sure the collection exists.
# If the collection exists, then item does not.
return false if redis_set_cache.exist?(name)
redis_set_cache.include?(name, value)
__send__(name).include?(value) # rubocop:disable GitlabSecurity/PublicSend
end
end
......
......@@ -25,7 +25,7 @@ module Gitlab
end
def read(key)
with { |redis| redis.smembers(cache_key(key)) }
with { |redis| redis.sscan_each(cache_key(key)).to_a }
end
def write(key, value)
......@@ -47,11 +47,10 @@ module Gitlab
end
def fetch(key, &block)
if exist?(key)
read(key)
else
write(key, yield)
end
result = read(key)
return result unless result.empty?
write(key, yield)
end
def include?(key, value)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment