• Paul E. McKenney's avatar
    rcu-tasks: Fix computation of CPU-to-list shift counts · da123016
    Paul E. McKenney authored
    The ->percpu_enqueue_shift field is used to map from the running CPU
    number to the index of the corresponding callback list.  This mapping
    can change at runtime in response to varying callback load, resulting
    in varying levels of contention on the callback-list locks.
    
    Unfortunately, the initial value of this field is correct only if the
    system happens to have a power-of-two number of CPUs, otherwise the
    callbacks from the high-numbered CPUs can be placed into the callback list
    indexed by 1 (rather than 0), and those index-1 callbacks will be ignored.
    This can result in soft lockups and hangs.
    
    This commit therefore corrects this mapping, adding one to this shift
    count as needed for systems having odd numbers of CPUs.
    
    Fixes: 7a30871b ("rcu-tasks: Introduce ->percpu_enqueue_shift for dynamic queue selection")
    Reported-by: default avatarAndrii Nakryiko <andrii.nakryiko@gmail.com>
    Cc: Reported-by: Martin Lau <kafai@fb.com>
    Cc: Neeraj Upadhyay <neeraj.iitr10@gmail.com>
    Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
    da123016
tasks.h 58.5 KB