Commit 580fc82e authored by Stan Hu's avatar Stan Hu

Extend gRPC timeouts for Rake tasks

In https://gitlab.com/gitlab-org/gitlab/merge_requests/16926 we added
gRPC timeouts for calls that did not previously have timeouts to prevent
Sidekiq queries from getting stuck. In addition, we also made long
timeouts 55 seconds for non-Sidekiq requests, but this meant Rake tasks
also fell into this bucket. Rake backup tasks with large repositories
would fail because the CreateBundle RPC would time out after 55 seconds.

To avoid this trap, we flip the logic of long_timeout: instead of
checking for Sidekiq (or other background jobs), we only lower the
timeout to 55 seconds if we're servicing a Web request in Puma or
Unicorn.

Closes https://gitlab.com/gitlab-org/gitlab/issues/22398
parent 0a4a4ba2
---
title: Extend gRPC timeouts for Rake tasks
merge_request: 19461
author:
type: fixed
......@@ -383,13 +383,17 @@ module Gitlab
end
def self.long_timeout
if Sidekiq.server?
6.hours
else
if web_app_server?
default_timeout
else
6.hours
end
end
def self.web_app_server?
defined?(::Unicorn) || defined?(::Puma)
end
def self.storage_metadata_file_path(storage)
Gitlab::GitalyClient::StorageSettings.allow_disk_access do
File.join(
......
......@@ -17,6 +17,28 @@ describe Gitlab::GitalyClient do
})
end
describe '.long_timeout' do
context 'default case' do
it { expect(subject.long_timeout).to eq(6.hours) }
end
context 'running in Unicorn' do
before do
stub_const('Unicorn', 1)
end
it { expect(subject.long_timeout).to eq(55) }
end
context 'running in Puma' do
before do
stub_const('Puma', 1)
end
it { expect(subject.long_timeout).to eq(55) }
end
end
describe '.filesystem_id_from_disk' do
it 'catches errors' do
[Errno::ENOENT, Errno::EACCES, JSON::ParserError].each do |error|
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment