• Frederic Weisbecker's avatar
    tick/nohz: Narrow down noise while setting current task's tick dependency · 3c8920e2
    Frederic Weisbecker authored
    Setting a tick dependency on any task, including the case where a task
    sets that dependency on itself, triggers an IPI to all CPUs.  That is
    of course suboptimal but it had previously not been an issue because it
    was only used by POSIX CPU timers on nohz_full, which apparently never
    occurs in latency-sensitive workloads in production.  (Or users of such
    systems are suffering in silence on the one hand or venting their ire
    on the wrong people on the other.)
    
    But RCU now sets a task tick dependency on the current task in order
    to fix stall issues that can occur during RCU callback processing.
    Thus, RCU callback processing triggers frequent system-wide IPIs from
    nohz_full CPUs.  This is quite counter-productive, after all, avoiding
    IPIs is what nohz_full is supposed to be all about.
    
    This commit therefore optimizes tasks' self-setting of a task tick
    dependency by using tick_nohz_full_kick() to avoid the system-wide IPI.
    Instead, only the execution of the one task is disturbed, which is
    acceptable given that this disturbance is well down into the noise
    compared to the degree to which the RCU callback processing itself
    disturbs execution.
    
    Fixes: 6a949b7a (rcu: Force on tick when invoking lots of callbacks)
    Reported-by: default avatarMatt Fleming <matt@codeblueprint.co.uk>
    Signed-off-by: default avatarFrederic Weisbecker <frederic@kernel.org>
    Cc: stable@kernel.org
    Cc: Paul E. McKenney <paulmck@kernel.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Ingo Molnar <mingo@kernel.org>
    Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
    3c8920e2
tick-sched.c 35.5 KB