• Alexandru Croitor's avatar
    Rebalance issues relative position without transaction · 8834fda8
    Alexandru Croitor authored
    Rebalancing issues in a long running transaction with a lock
    retry generates subtransactions that lead to overall DB
    performance degradations.
    
    see https://gitlab.com/gitlab-org/gitlab/-/issues/338346
    
    So we are moving issues rebalancing out of one single big
    transaction by locking the repositioning within a project or
    namespace during the rebalance.
    
    Rebalancing is a long running job, it can take multiple hours
    to finish a rebalance of a namespace with large number of issues.
    This change introduces a simple mechanism of resuming rebalance
    from a checkpoint.
    
    - A limited number of rebalancing jobs are allowed to run,
    5 at this point
    - Before starting a rebalance the number of running rebalances
    is checked.
    - If the limit of running rebalances is met we store the first
    project id from the list of projects to be rebalanced, use that
    to determine how many rebalances are running
    - We load all namespace issue ids into a redis sorted set, by
    using current i...
    8834fda8
issue.rb 20.9 KB