• Paul E. McKenney's avatar
    rcu/nocb: Avoid ->nocb_lock capture by corresponding CPU · 81c0b3d7
    Paul E. McKenney authored
    A given rcu_data structure's ->nocb_lock can be acquired very frequently
    by the corresponding CPU and occasionally by the corresponding no-CBs
    grace-period and callbacks kthreads.  In particular, these two kthreads
    will have frequent gaps between ->nocb_lock acquisitions that are roughly
    a grace period in duration.  This means that any excessive ->nocb_lock
    contention will be due to the CPU's acquisitions, and this in turn
    enables a very naive contention-avoidance strategy to be quite effective.
    
    This commit therefore modifies rcu_nocb_lock() to first
    attempt a raw_spin_trylock(), and to atomically increment a
    separate ->nocb_lock_contended across a raw_spin_lock().  This new
    ->nocb_lock_contended field is checked in __call_rcu_nocb_wake() when
    interrupts are enabled, with a spin-wait for contending acquisitions
    to complete, thus allowing the kthreads a chance to acquire the lock.
    Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
    81c0b3d7
tree.h 18.3 KB