• Frederic Weisbecker's avatar
    rcu/nocb: Don't invoke local rcu core on callback overload from nocb kthread · 0598a4d4
    Frederic Weisbecker authored
    rcu_core() tries to ensure that its self-invocation in case of callbacks
    overload only happen in softirq/rcuc mode. Indeed it doesn't make sense
    to trigger local RCU core from nocb_cb kthread since it can execute
    on a CPU different from the target rdp. Also in case of overload, the
    nocb_cb kthread simply iterates a new loop of callbacks processing.
    
    However the "offloaded" check that aims at preventing misplaced
    rcu_core() invocations is wrong. First of all that state is volatile
    and second: softirq/rcuc can execute while the target rdp is offloaded.
    As a result rcu_core() can be invoked on the wrong CPU while in the
    process of (de-)offloading.
    
    Fix that with moving the rcu_core() self-invocation to rcu_core() itself,
    irrespective of the rdp offloaded state.
    Tested-by: default avatarValentin Schneider <valentin.schneider@arm.com>
    Tested-by: default avatarSebastian Andrzej Siewior <bigeasy@linutronix.de>
    Signed-off-by: default avatarFrederic Weisbecker <frederic@kernel.org>
    Cc: Valentin Schneider <valentin.schneider@arm.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
    Cc: Josh Triplett <josh@joshtriplett.org>
    Cc: Joel Fernandes <joel@joelfernandes.org>
    Cc: Boqun Feng <boqun.feng@gmail.com>
    Cc: Neeraj Upadhyay <neeraju@codeaurora.org>
    Cc: Uladzislau Rezki <urezki@gmail.com>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
    0598a4d4
tree.c 153 KB