• Huaixin Chang's avatar
    sched/fair: Fix race between runtime distribution and assignment · 26a8b127
    Huaixin Chang authored
    Currently, there is a potential race between distribute_cfs_runtime()
    and assign_cfs_rq_runtime(). Race happens when cfs_b->runtime is read,
    distributes without holding lock and finds out there is not enough
    runtime to charge against after distribution. Because
    assign_cfs_rq_runtime() might be called during distribution, and use
    cfs_b->runtime at the same time.
    
    Fibtest is the tool to test this race. Assume all gcfs_rq is throttled
    and cfs period timer runs, slow threads might run and sleep, returning
    unused cfs_rq runtime and keeping min_cfs_rq_runtime in their local
    pool. If all this happens sufficiently quickly, cfs_b->runtime will drop
    a lot. If runtime distributed is large too, over-use of runtime happens.
    
    A runtime over-using by about 70 percent of quota is seen when we
    test fibtest on a 96-core machine. We run fibtest with 1 fast thread and
    95 slow threads in test group, configure 10ms quota for this group and
    see the CPU usage of fibtest is 17.0%, which is far more than the
    expected 10%.
    
    On a smaller machine with 32 cores, we also run fibtest with 96
    threads. CPU usage is more than 12%, which is also more than expected
    10%. This shows that on similar workloads, this race do affect CPU
    bandwidth control.
    
    Solve this by holding lock inside distribute_cfs_runtime().
    
    Fixes: c06f04c7 ("sched: Fix potential near-infinite distribute_cfs_runtime() loop")
    Reviewed-by: default avatarBen Segall <bsegall@google.com>
    Signed-off-by: default avatarHuaixin Chang <changhuaixin@linux.alibaba.com>
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
    Link: https://lore.kernel.org/lkml/20200325092602.22471-1-changhuaixin@linux.alibaba.com/
    26a8b127
fair.c 294 KB