This Merge request is to solve Deadlock issue which cause gitalb 502 response at some point because gitaly is fully busy.
Gitaly seems not well configured to handle an important amount of connections. After some time there is this traceback in production.log:
Gitlab::Git::CommandTimedOut (4:Deadline Exceeded.):
lib/gitlab/git/wraps_gitaly_errors.rb:13:in `rescue in wrapped_gitaly_errors'
lib/gitlab/git/wraps_gitaly_errors.rb:6:in `wrapped_gitaly_errors'
lib/gitlab/git/commit.rb:153:in `find_all'
app/models/network/graph.rb:123:in `find_commits'
app/models/network/graph.rb:88:in `count_to_display_commit_in_center'
app/models/network/graph.rb:47:in `block in collect_commits'
lib/gitlab/gitaly_client.rb:336:in `allow_n_plus_1_calls'
app/models/network/graph.rb:46:in `collect_commits'
app/models/network/graph.rb:18:in `initialize'
app/controllers/projects/network_controller.rb:30:in `new'
app/controllers/projects/network_controller.rb:30:in `block (2 levels) in show'
app/controllers/projects/network_controller.rb:22:in `show'
ee/lib/gitlab/ip_address_state.rb:10:in `with'
ee/app/controllers/ee/application_controller.rb:45:in `set_current_ip_address'
app/controllers/application_controller.rb:524:in `set_current_admin'
lib/gitlab/session.rb:11:in `with_session'
app/controllers/application_controller.rb:515:in `set_session_storage'
lib/gitlab/i18n.rb:107:in `with_locale'
app/controllers/application_controller.rb:508:in `set_locale'
app/controllers/application_controller.rb:499:in `set_current_context'
lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call'
lib/gitlab/middleware/memory_report.rb:13:in `call'
lib/gitlab/middleware/speedscope.rb:13:in `call'
lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call'
lib/gitlab/middleware/rails_queue_duration.rb:33:in `call'
lib/gitlab/metrics/rack_middleware.rb:16:in `block in call'
lib/gitlab/metrics/web_transaction.rb:46:in `run'
lib/gitlab/metrics/rack_middleware.rb:16:in `call'
lib/gitlab/jira/middleware.rb:19:in `call'
lib/gitlab/middleware/go.rb:20:in `call'
lib/gitlab/etag_caching/middleware.rb:21:in `call'
lib/gitlab/middleware/query_analyzer.rb:11:in `block in call'
lib/gitlab/database/query_analyzer.rb:37:in `within'
lib/gitlab/middleware/query_analyzer.rb:11:in `call'
lib/gitlab/middleware/multipart.rb:173:in `call'
lib/gitlab/middleware/read_only/controller.rb:50:in `call'
lib/gitlab/middleware/read_only.rb:18:in `call'
lib/gitlab/middleware/same_site_cookies.rb:27:in `call'
lib/gitlab/middleware/basic_health_check.rb:25:in `call'
lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'
lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'
lib/gitlab/middleware/request_context.rb:21:in `call'
lib/gitlab/middleware/webhook_recursion_detection.rb:15:in `call'
config/initializers/fix_local_cache_middleware.rb:11:in `call'
lib/gitlab/middleware/compressed_json.rb:37:in `call'
lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call'
lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:79:in `call'
lib/gitlab/middleware/release_env.rb:13:in `call'
We can now configure gitaly concurency according to the usage we need.