• Ken Chen's avatar
    sched: fix inconsistency when redistribute per-cpu tg->cfs_rq shares · ec4e0e2f
    Ken Chen authored
    Impact: make load-balancing more consistent
    
    In the update_shares() path leading to tg_shares_up(), the calculation of
    per-cpu cfs_rq shares is rather erratic even under moderate task wake up
    rate.  The problem is that the per-cpu tg->cfs_rq load weight used in the
    sd_rq_weight aggregation and actual redistribution of the cfs_rq->shares
    are collected at different time.  Under moderate system load, we've seen
    quite a bit of variation on the cfs_rq->shares and ultimately wildly
    affects sched_entity's load weight.
    
    This patch caches the result of initial per-cpu load weight when doing the
    sum calculation, and then pass it down to update_group_shares_cpu() for
    redistributing per-cpu cfs_rq shares.  This allows consistent total cfs_rq
    shares across all CPUs. It also simplifies the rounding and zero load
    weight check.
    Signed-off-by: default avatarKen Chen <kenchen@google.com>
    Acked-by: default avatarPeter Zijlstra <a.p.zijlstra@chello.nl>
    Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
    ec4e0e2f
sched.c 224 KB