• Paul E. McKenney's avatar
    rcu-tasks: Eliminate RCU Tasks Trace IPIs to online CPUs · e386b672
    Paul E. McKenney authored
    Currently, the RCU Tasks Trace grace-period kthread IPIs each online CPU
    using smp_call_function_single() in order to track any tasks currently in
    RCU Tasks Trace read-side critical sections during which the corresponding
    task has neither blocked nor been preempted.  These IPIs are annoying
    and are also not strictly necessary because any task that blocks or is
    preempted within its current RCU Tasks Trace read-side critical section
    will be tracked on one of the per-CPU rcu_tasks_percpu structure's
    ->rtp_blkd_tasks list.  So the only time that this is a problem is if
    one of the CPUs runs through a long-duration RCU Tasks Trace read-side
    critical section without a context switch.
    
    Note that the task_call_func() function cannot help here because there is
    no safe way to identify the target task.  Of course, the task_call_func()
    function will be very useful later, when processing the list of tasks,
    but it needs to know the task.
    
    This commit therefore creates a cpu_curr_snapshot() function that returns
    a pointer the task_struct structure of some task that happened to be
    running on the specified CPU more or less during the time that the
    cpu_curr_snapshot() function was executing.  If there was no context
    switch during this time, this function will return a pointer to the
    task_struct structure of the task that was running throughout.  If there
    was a context switch, then the outgoing task will be taken care of by
    RCU's context-switch hook, and the incoming task was either already taken
    care during some previous context switch, or it is not currently within an
    RCU Tasks Trace read-side critical section.  And in this latter case, the
    grace period already started, so there is no need to wait on this task.
    
    This new cpu_curr_snapshot() function is invoked on each CPU early in
    the RCU Tasks Trace grace-period processing, and the resulting tasks
    are queued for later quiescent-state inspection.
    Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
    Cc: Eric Dumazet <edumazet@google.com>
    Cc: Alexei Starovoitov <ast@kernel.org>
    Cc: Andrii Nakryiko <andrii@kernel.org>
    Cc: Martin KaFai Lau <kafai@fb.com>
    Cc: KP Singh <kpsingh@kernel.org>
    e386b672
core.c 284 KB