Commit ba9b4c4d authored by Sean McGivern's avatar Sean McGivern

Avoid hitting statement timeout finding MR pipelines

For MRs with many thousands of commits, `SELECT DISTINCT(sha)` will be very
slow.

What we can't do to fix this:

1. Add an index. Postgres won't use it for DISTINCT without a lot of ceremony.
2. Do the `uniq` in Ruby. That can still be very slow with hundreds of
   thousands of commits.
3. Use a subquery. We haven't removed the `st_commits` column yet, but we will
   soon.

Until 3 is available to us, we can just do 2, but also add a limit clause. There
is no ordering, so this may return different results, but our goal with these
MRs is just to get them to load, so it's not a huge deal.
parent e9d35261
......@@ -874,7 +874,7 @@ class MergeRequest < ActiveRecord::Base
#
def all_commit_shas
if persisted?
column_shas = MergeRequestDiffCommit.where(merge_request_diff: merge_request_diffs).pluck('DISTINCT(sha)')
column_shas = MergeRequestDiffCommit.where(merge_request_diff: merge_request_diffs).limit(10_000).pluck('sha')
serialised_shas = merge_request_diffs.where.not(st_commits: nil).flat_map(&:commit_shas)
(column_shas + serialised_shas).uniq
......
---
title: Stop merge requests with thousands of commits from timing out
merge_request: 15063
author:
type: performance
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment