1. 21 Jun, 2022 3 commits
    • Paul E. McKenney's avatar
      rcu-tasks: Eliminate RCU Tasks Trace IPIs to online CPUs · e386b672
      Paul E. McKenney authored
      Currently, the RCU Tasks Trace grace-period kthread IPIs each online CPU
      using smp_call_function_single() in order to track any tasks currently in
      RCU Tasks Trace read-side critical sections during which the corresponding
      task has neither blocked nor been preempted.  These IPIs are annoying
      and are also not strictly necessary because any task that blocks or is
      preempted within its current RCU Tasks Trace read-side critical section
      will be tracked on one of the per-CPU rcu_tasks_percpu structure's
      ->rtp_blkd_tasks list.  So the only time that this is a problem is if
      one of the CPUs runs through a long-duration RCU Tasks Trace read-side
      critical section without a context switch.
      
      Note that the task_call_func() function cannot help here because there is
      no safe way to identify the target task.  Of course, the task_call_func()
      function will be very useful later, when processing the list of tasks,
      but it needs to know the task.
      
      This commit therefore creates a cpu_curr_snapshot() function that returns
      a pointer the task_struct structure of some task that happened to be
      running on the specified CPU more or less during the time that the
      cpu_curr_snapshot() function was executing.  If there was no context
      switch during this time, this function will return a pointer to the
      task_struct structure of the task that was running throughout.  If there
      was a context switch, then the outgoing task will be taken care of by
      RCU's context-switch hook, and the incoming task was either already taken
      care during some previous context switch, or it is not currently within an
      RCU Tasks Trace read-side critical section.  And in this latter case, the
      grace period already started, so there is no need to wait on this task.
      
      This new cpu_curr_snapshot() function is invoked on each CPU early in
      the RCU Tasks Trace grace-period processing, and the resulting tasks
      are queued for later quiescent-state inspection.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      e386b672
    • Paul E. McKenney's avatar
      rcu-tasks: Maintain a count of tasks blocking RCU Tasks Trace grace period · ffcc21a3
      Paul E. McKenney authored
      This commit maintains a new n_trc_holdouts counter that tracks the number
      of tasks blocking the RCU Tasks grace period.  This counter is useful
      for debugging, and its value has been added to a diagostic message.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      ffcc21a3
    • Paul E. McKenney's avatar
      rcu-tasks: Stop RCU Tasks Trace from scanning full tasks list · 1a4a8153
      Paul E. McKenney authored
      This commit takes off the training wheels and relies only on scanning
      currently running tasks and tasks that have blocked or been preempted
      within their current RCU Tasks Trace read-side critical section.
      
      Before this commit, the time complexity of an RCU Tasks Trace grace
      period is O(T), where T is the number of tasks.  After this commit,
      this time complexity is O(C+B), where C is the number of CPUs and B
      is the number of tasks that have blocked (or been preempted) at least
      once during their current RCU Tasks Trace read-side critical sections.
      Of course, if all tasks have blocked (or been preempted) at least once
      during their current RCU Tasks Trace read-side critical sections, this is
      still O(T), but current expectations are that RCU Tasks Trace read-side
      critical section will be short and that there will normally not be large
      numbers of tasks blocked within such a critical section.
      
      Dave Marchevsky kindly measured the effects of this commit on the RCU
      Tasks Trace grace-period latency and the rcu_tasks_trace_kthread task's
      CPU consumption per RCU Tasks Trace grace period over the course of a
      fixed test, all in milliseconds:
      
      		Before			After
      
      GP latency	22.3 ms stddev > 0.1	17.0 ms stddev < 0.1
      
      GP CPU		 2.3 ms stddev 0.3	 1.1 ms stddev 0.2
      
      This was on a system with 15,000 tasks, so it is reasonable to expect
      much larger savings on the systems on which this issue was first noted,
      given that they sport well in excess of 100,000 tasks.  CPU consumption
      was measured using profiling techniques.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      Tested-by: default avatarDave Marchevsky <davemarchevsky@fb.com>
      1a4a8153
  2. 20 Jun, 2022 25 commits
    • Paul E. McKenney's avatar
      rcu-tasks: Stop RCU Tasks Trace from scanning idle tasks · 955a0192
      Paul E. McKenney authored
      Now that RCU scans both running tasks and tasks that have blocked within
      their current RCU Tasks Trace read-side critical section, there is no
      need for it to scan the idle tasks.  After all, an idle loop should not
      be remain within an RCU Tasks Trace read-side critical section across
      exit from idle, and from a BPF viewpoint, functions invoked from the
      idle loop should not sleep.  So only running idle tasks can be within
      RCU Tasks Trace read-side critical sections.
      
      This commit therefore removes the scan of the idle tasks from the
      rcu_tasks_trace_postscan() function.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      955a0192
    • Paul E. McKenney's avatar
      rcu-tasks: Pull in tasks blocked within RCU Tasks Trace readers · dc7d54b4
      Paul E. McKenney authored
      This commit scans each CPU's ->rtp_blkd_tasks list, adding them to
      the list of holdout tasks.  This will cause the current RCU Tasks Trace
      grace period to wait until these tasks exit their RCU Tasks Trace
      read-side critical sections.  This commit will enable later work
      omitting the scan of the full task list.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      dc7d54b4
    • Paul E. McKenney's avatar
      rcu-tasks: Scan running tasks for RCU Tasks Trace readers · 7460ade1
      Paul E. McKenney authored
      A running task might be within an RCU Tasks Trace read-side critical
      section for any length of time, but will not be placed on any of the
      per-CPU rcu_tasks_percpu structure's ->rtp_blkd_tasks lists.  Therefore
      any RCU Tasks Trace grace-period processing that does not scan the full
      task list must interact with the running tasks.
      
      This commit therefore causes the rcu_tasks_trace_pregp_step() function
      to IPI each CPU in order to place the corresponding task on the holdouts
      list and to record whether or not it was in an RCU Tasks Trace read-side
      critical section.  Yes, it is possible to avoid adding it to that list
      if it is not a reader, but that would prevent the system from remembering
      that this task was in a quiescent state.  Which is why the running tasks
      are unconditionally added to the holdout list.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      7460ade1
    • Paul E. McKenney's avatar
      rcu-tasks: Avoid rcu_tasks_trace_pertask() duplicate list additions · 19415004
      Paul E. McKenney authored
      This commit adds checks within rcu_tasks_trace_pertask() to avoid
      duplicate (and destructive) additions to the holdouts list.  These checks
      will be required later due to the possibility of a given task having
      blocked while in an RCU Tasks Trace read-side critical section, but now
      running on a CPU.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      19415004
    • Paul E. McKenney's avatar
      rcu-tasks: Move rcu_tasks_trace_pertask() before rcu_tasks_trace_pregp_step() · 1fa98e2e
      Paul E. McKenney authored
      This is a code-motion-only commit that moves rcu_tasks_trace_pertask()
      to precede rcu_tasks_trace_pregp_step(), so that the latter will be
      able to invoke the other without forward references.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      1fa98e2e
    • Paul E. McKenney's avatar
      rcu-tasks: Add blocked-task indicator to RCU Tasks Trace stall warnings · 387c0ad7
      Paul E. McKenney authored
      This commit adds a "B" indicator to the RCU Tasks Trace CPU stall warning
      when the task has blocked within its current read-side critical section.
      This serves as a debugging aid.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      387c0ad7
    • Paul E. McKenney's avatar
      rcu-tasks: Untrack blocked RCU Tasks Trace at reader end · 0bcb3868
      Paul E. McKenney authored
      This commit causes rcu_read_unlock_trace() to check for the current
      task being on a per-CPU list within the rcu_tasks_percpu structure,
      and removes it from that list if so.  This has the effect of curtailing
      tracking of a task that blocked within an RCU Tasks Trace read-side
      critical section once it exits that critical section.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      0bcb3868
    • Paul E. McKenney's avatar
      rcu-tasks: Track blocked RCU Tasks Trace readers · 0356d4e6
      Paul E. McKenney authored
      This commit places any task that has ever blocked within its current
      RCU Tasks Trace read-side critical section on a per-CPU list within the
      rcu_tasks_percpu structure.  Tasks are removed from this list when they
      exit by the exit_tasks_rcu_finish_trace() function.  The purpose of this
      commit is to provide the information needed to eliminate the current
      scan of the full task list.
      
      This commit offsets the INT_MIN value for ->trc_reader_nesting with the
      new nesting level in order to avoid queueing tasks that are exiting
      their read-side critical sections.
      
      [ paulmck: Apply kernel test robot feedback. ]
      [ paulmck: Apply feedback from syzbot+9bb26e7c5e8e4fa7e641@syzkaller.appspotmail.com ]
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Tested-by: default avatarsyzbot <syzbot+9bb26e7c5e8e4fa7e641@syzkaller.appspotmail.com>
      Tested-by: default avatar"Zhang, Qiang1" <qiang1.zhang@intel.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      0356d4e6
    • Paul E. McKenney's avatar
      rcu-tasks: Add data structures for lightweight grace periods · 434c9eef
      Paul E. McKenney authored
      This commit adds fields to task_struct and to rcu_tasks_percpu that will
      be used to avoid the task-list scan for RCU Tasks Trace grace periods,
      and also initializes these fields.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      434c9eef
    • Paul E. McKenney's avatar
      rcu-tasks: Make RCU Tasks Trace stall warning handle idle offline tasks · f90f19da
      Paul E. McKenney authored
      When a CPU is offline, its idle task can appear to be running, but it
      cannot be doing anything while CPU-hotplug operations are excluded.
      This commit takes advantage of that fact by making trc_check_slow_task()
      check for task_curr(t) && cpu_online(task_cpu(t)), and recording
      full information in that case.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      f90f19da
    • Paul E. McKenney's avatar
      rcu-tasks: Make RCU Tasks Trace stall warnings print full .b.need_qs field · be15a164
      Paul E. McKenney authored
      Currently, the RCU Tasks Trace CPU stall warning simply indicates
      whether or not the .b.need_qs field is zero.  This commit shows the
      three permitted values and flags other values with either "!" or "?".
      This is a debugging aid.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      be15a164
    • Paul E. McKenney's avatar
      rcu-tasks: Flag offline CPUs in RCU Tasks Trace stall warnings · c8c03ad9
      Paul E. McKenney authored
      This commit tags offline CPUs with "(offline)" in RCU Tasks Trace CPU
      stall warnings.  This is a debugging aid.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      c8c03ad9
    • Paul E. McKenney's avatar
      rcu-tasks: Add slow-IPI indicator to RCU Tasks Trace stall warnings · 9f3eb5fb
      Paul E. McKenney authored
      This commit adds a "I" indicator to the RCU Tasks Trace CPU stall
      warning when an IPI directed to a task has thus far failed to arrive.
      This serves as a debugging aid.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      9f3eb5fb
    • Paul E. McKenney's avatar
      rcu-tasks: Simplify trc_inspect_reader() QS logic · 0968e892
      Paul E. McKenney authored
      Currently, trc_inspect_reader() does one check for nesting less than
      or equal to zero, then sorts out the distinctions within this single
      "if" statement.  This commit simplifies the logic by providing one
      "if" statement for quiescent states (nesting of zero) and another "if"
      statement for transitioning from one nesting level to another or the
      outermost rcu_read_unlock_trace() (negative nesting).
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      0968e892
    • Paul E. McKenney's avatar
      rcu-tasks: Make rcu_note_context_switch() unconditionally call rcu_tasks_qs() · 6a694411
      Paul E. McKenney authored
      This commit makes rcu_note_context_switch() unconditionally invoke the
      rcu_tasks_qs() function, as opposed to doing so only when RCU (as opposed
      to RCU Tasks Trace) urgently needs a grace period to end.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      6a694411
    • Paul E. McKenney's avatar
      rcu-tasks: RCU Tasks Trace grace-period kthread has implicit QS · 5d4c90d7
      Paul E. McKenney authored
      Because the task driving the grace-period kthread is in quiescent state
      throughout, this commit excludes it from the list of tasks from which
      a quiescent state is needed.
      
      This does mean that attaching a sleepable BPF program to function in
      kernel/rcu/tasks.h is a bad idea, by the way.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      5d4c90d7
    • Paul E. McKenney's avatar
      rcu-tasks: Handle idle tasks for recently offlined CPUs · 897ba84d
      Paul E. McKenney authored
      This commit identifies idle tasks for recently offlined CPUs as residing
      in a quiescent state.  This is safe only because CPU-hotplug operations
      are excluded during these checks.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      897ba84d
    • Paul E. McKenney's avatar
      rcu-tasks: Idle tasks on offline CPUs are in quiescent states · 5c9a9ca4
      Paul E. McKenney authored
      Any idle task corresponding to an offline CPU is in an RCU Tasks Trace
      quiescent state.  This commit causes rcu_tasks_trace_postscan() to ignore
      idle tasks for offline CPUs, which it can do safely due to CPU-hotplug
      operations being disabled.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      5c9a9ca4
    • Paul E. McKenney's avatar
      rcu-tasks: Make trc_read_check_handler() fetch ->trc_reader_nesting only once · 9ff86b4c
      Paul E. McKenney authored
      This commit replaces the pair of READ_ONCE(t->trc_reader_nesting) calls
      with a single such call and a local variable.  This makes the code's
      intent more clear.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      9ff86b4c
    • Paul E. McKenney's avatar
      rcu-tasks: Remove rcu_tasks_trace_postgp() wait for counter · 55061126
      Paul E. McKenney authored
      Now that tasks are not removed from the list until they have responded to
      any needed request for a quiescent state, it is no longer necessary to
      wait for the trc_n_readers_need_end counter to go to zero.  This commit
      therefore removes that waiting code.
      
      It is therefore also no longer necessary for rcu_tasks_trace_postgp() to
      do the final decrement of this counter, so that code is also removed.
      This in turn means that trc_n_readers_need_end counter itself can
      be removed, as can the rcu_tasks_trace_iw irq_work structure and the
      rcu_read_unlock_iw() function.
      
      [ paulmck: Apply feedback from Zqiang. ]
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      55061126
    • Paul E. McKenney's avatar
      rcu-tasks: Merge state into .b.need_qs and atomically update · 3847b645
      Paul E. McKenney authored
      This commit gets rid of the task_struct structure's ->trc_reader_checked
      field, making it instead be a bit within the task_struct structure's
      existing ->trc_reader_special.b.need_qs field.  This commit also
      atomically loads, stores, and checks the resulting combination of the
      reader-checked and need-quiescent state flags.  This will in turn allow
      significant simplification of the rcu_tasks_trace_postgp() function
      as well as elimination of the trc_n_readers_need_end counter in later
      commits.  These changes will in turn simplify later elimination of the
      RCU Tasks Trace scan of the task list, which will make RCU Tasks Trace
      grace periods less CPU-intensive.
      
      [ paulmck: Apply kernel test robot feedback. ]
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Andrii Nakryiko <andrii@kernel.org>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: KP Singh <kpsingh@kernel.org>
      3847b645
    • Paul E. McKenney's avatar
      rcu-tasks: Drive synchronous grace periods from calling task · 4a8cc433
      Paul E. McKenney authored
      This commit causes synchronous grace periods to be driven from the task
      invoking synchronize_rcu_*(), allowing these functions to be invoked from
      the mid-boot dead zone extending from when the scheduler was initialized
      to to point that the various RCU tasks grace-period kthreads are spawned.
      This change will allow the self-tests to run in a consistent manner.
      Reported-by: default avatarMatthew Wilcox <willy@infradead.org>
      Reported-by: default avatarZhouyi Zhou <zhouzhouyi@gmail.com>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      4a8cc433
    • Paul E. McKenney's avatar
      rcu-tasks: Move synchronize_rcu_tasks_generic() down · 68cb4720
      Paul E. McKenney authored
      This is strictly a code-motion commit that moves the
      synchronize_rcu_tasks_generic() down to where it can invoke
      rcu_tasks_one_gp() without the need for a forward declaration.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      68cb4720
    • Paul E. McKenney's avatar
      rcu-tasks: Split rcu_tasks_one_gp() from rcu_tasks_kthread() · d96225fd
      Paul E. McKenney authored
      This commit abstracts most of the rcu_tasks_kthread() function's loop
      body into a new rcu_tasks_one_gp() function.  It also introduces
      a new ->tasks_gp_mutex to synchronize concurrent calls to this new
      rcu_tasks_one_gp() function.  This commit is preparation for allowing
      RCU tasks grace periods to be driven by the calling task during the
      mid-boot dead zone.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      d96225fd
    • Paul E. McKenney's avatar
      rcu-tasks: Check for abandoned callbacks · 4cf0585c
      Paul E. McKenney authored
      This commit adds a debugging scan for callbacks that got lost during a
      callback-queueing transition.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      4cf0585c
  3. 19 Jun, 2022 11 commits
    • Linus Torvalds's avatar
      Linux 5.19-rc3 · a111daf0
      Linus Torvalds authored
      a111daf0
    • Linus Torvalds's avatar
      Merge tag 'x86-urgent-2022-06-19' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · 05c6ca85
      Linus Torvalds authored
      Pull x86 fixes from Thomas Gleixner:
      
       - Make RESERVE_BRK() work again with older binutils. The recent
         'simplification' broke that.
      
       - Make early #VE handling increment RIP when successful.
      
       - Make the #VE code consistent vs. the RIP adjustments and add
         comments.
      
       - Handle load_unaligned_zeropad() across page boundaries correctly in
         #VE when the second page is shared.
      
      * tag 'x86-urgent-2022-06-19' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
        x86/tdx: Handle load_unaligned_zeropad() page-cross to a shared page
        x86/tdx: Clarify RIP adjustments in #VE handler
        x86/tdx: Fix early #VE handling
        x86/mm: Fix RESERVE_BRK() for older binutils
      05c6ca85
    • Linus Torvalds's avatar
      Merge tag 'objtool-urgent-2022-06-19' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · 5d770f11
      Linus Torvalds authored
      Pull build tooling updates from Thomas Gleixner:
      
       - Remove obsolete CONFIG_X86_SMAP reference from objtool
      
       - Fix overlapping text section failures in faddr2line for real
      
       - Remove OBJECT_FILES_NON_STANDARD usage from x86 ftrace and replace it
         with finegrained annotations so objtool can validate that code
         correctly.
      
      * tag 'objtool-urgent-2022-06-19' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
        x86/ftrace: Remove OBJECT_FILES_NON_STANDARD usage
        faddr2line: Fix overlapping text section failures, the sequel
        objtool: Fix obsolete reference to CONFIG_X86_SMAP
      5d770f11
    • Linus Torvalds's avatar
      Merge tag 'sched-urgent-2022-06-19' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · 727c3991
      Linus Torvalds authored
      Pull scheduler fix from Thomas Gleixner:
       "A single scheduler fix plugging a race between sched_setscheduler()
        and balance_push().
      
        sched_setscheduler() spliced the balance callbacks accross a lock
        break which makes it possible for an interleaving schedule() to
        observe an empty list"
      
      * tag 'sched-urgent-2022-06-19' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
        sched: Fix balance_push() vs __sched_setscheduler()
      727c3991
    • Linus Torvalds's avatar
      Merge tag 'locking-urgent-2022-06-19' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · 4afb6515
      Linus Torvalds authored
      Pull lockdep fix from Thomas Gleixner:
       "A RT fix for lockdep.
      
        lockdep invokes prandom_u32() to create cookies. This worked until
        prandom_u32() was switched to the real random generator, which takes a
        spinlock for extraction, which does not work on RT when invoked from
        atomic contexts.
      
        lockdep has no requirement for real random numbers and it turns out
        sched_clock() is good enough to create the cookie. That works
        everywhere and is faster"
      
      * tag 'locking-urgent-2022-06-19' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
        locking/lockdep: Use sched_clock() for random numbers
      4afb6515
    • Linus Torvalds's avatar
      Merge tag 'irq-urgent-2022-06-19' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · 36da9f5f
      Linus Torvalds authored
      Pull irq fixes from Thomas Gleixner:
       "A set of interrupt subsystem updates:
      
        Core:
      
         - Ensure runtime power management for chained interrupts
      
        Drivers:
      
         - A collection of OF node refcount fixes
      
         - Unbreak MIPS uniprocessor builds
      
         - Fix xilinx interrupt controller Kconfig dependencies
      
         - Add a missing compatible string to the Uniphier driver"
      
      * tag 'irq-urgent-2022-06-19' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
        irqchip/loongson-liointc: Use architecture register to get coreid
        irqchip/uniphier-aidet: Add compatible string for NX1 SoC
        dt-bindings: interrupt-controller/uniphier-aidet: Add bindings for NX1 SoC
        irqchip/realtek-rtl: Fix refcount leak in map_interrupts
        irqchip/gic-v3: Fix refcount leak in gic_populate_ppi_partitions
        irqchip/gic-v3: Fix error handling in gic_populate_ppi_partitions
        irqchip/apple-aic: Fix refcount leak in aic_of_ic_init
        irqchip/apple-aic: Fix refcount leak in build_fiq_affinity
        irqchip/gic/realview: Fix refcount leak in realview_gic_of_init
        irqchip/xilinx: Remove microblaze+zynq dependency
        genirq: PM: Use runtime PM for chained interrupts
      36da9f5f
    • Linus Torvalds's avatar
      Merge tag 'char-misc-5.19-rc3-take2' of... · bc94632c
      Linus Torvalds authored
      Merge tag 'char-misc-5.19-rc3-take2' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc
      
      Pull char/misc driver fixes for real from Greg KH:
       "Let's tag the proper branch this time...
      
        Here are some small char/misc driver fixes for 5.19-rc3 that resolve
        some reported issues.
      
        They include:
      
         - mei driver fixes
      
         - comedi driver fix
      
         - rtsx build warning fix
      
         - fsl-mc-bus driver fix
      
        All of these have been in linux-next for a while with no reported
        issues"
      
      This is what the merge in commit f0ec9c65 _should_ have merged, but
      Greg fat-fingered the pull request and I got some small changes from
      linux-next instead there. Credit to Nathan Chancellor for eagle-eyes.
      
      Link: https://lore.kernel.org/all/Yqywy+Md2AfGDu8v@dev-arch.thelio-3990X/
      
      * tag 'char-misc-5.19-rc3-take2' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc:
        bus: fsl-mc-bus: fix KASAN use-after-free in fsl_mc_bus_remove()
        mei: me: add raptor lake point S DID
        mei: hbm: drop capability response on early shutdown
        mei: me: set internal pg flag to off on hardware reset
        misc: rtsx: Fix clang -Wsometimes-uninitialized in rts5261_init_from_hw()
        comedi: vmk80xx: fix expression for tx buffer size
      bc94632c
    • Linus Torvalds's avatar
      Merge tag 'i2c-for-5.19-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux · ee4eb6ee
      Linus Torvalds authored
      Pull i2c fixes from Wolfram Sang:
       "MAINTAINERS rectifications and a few minor driver fixes"
      
      * tag 'i2c-for-5.19-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux:
        i2c: mediatek: Fix an error handling path in mtk_i2c_probe()
        i2c: designware: Use standard optional ref clock implementation
        MAINTAINERS: core DT include belongs to core
        MAINTAINERS: add include/dt-bindings/i2c to I2C SUBSYSTEM HOST DRIVERS
        i2c: npcm7xx: Add check for platform_driver_register
        MAINTAINERS: Update Synopsys DesignWare I2C to Supported
      ee4eb6ee
    • Linus Torvalds's avatar
      Merge tag 'xfs-5.19-fixes-1' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux · 063232b6
      Linus Torvalds authored
      Pull xfs fixes from Darrick Wong:
       "There's not a whole lot this time around (I'm still on vacation) but
        here are some important fixes for new features merged in -rc1:
      
         - Fix a bug where inode flag changes would accidentally drop nrext64
      
         - Fix a race condition when toggling LARP mode"
      
      * tag 'xfs-5.19-fixes-1' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux:
        xfs: preserve DIFLAG2_NREXT64 when setting other inode attributes
        xfs: fix variable state usage
        xfs: fix TOCTOU race involving the new logged xattrs control knob
      063232b6
    • Linus Torvalds's avatar
      Merge tag 'ext4_for_linus_stable' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4 · 354c6e07
      Linus Torvalds authored
      Pull ext4 fixes from Ted Ts'o:
       "Fix a variety of bugs, many of which were found by folks using fuzzing
        or error injection.
      
        Also fix up how test_dummy_encryption mount option is handled for the
        new mount API.
      
        Finally, fix/cleanup a number of comments and ext4 Documentation
        files"
      
      * tag 'ext4_for_linus_stable' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4:
        ext4: fix a doubled word "need" in a comment
        ext4: add reserved GDT blocks check
        ext4: make variable "count" signed
        ext4: correct the judgment of BUG in ext4_mb_normalize_request
        ext4: fix bug_on ext4_mb_use_inode_pa
        ext4: fix up test_dummy_encryption handling for new mount API
        ext4: use kmemdup() to replace kmalloc + memcpy
        ext4: fix super block checksum incorrect after mount
        ext4: improve write performance with disabled delalloc
        ext4: fix warning when submitting superblock in ext4_commit_super()
        ext4, doc: remove unnecessary escaping
        ext4: fix incorrect comment in ext4_bio_write_page()
        fs: fix jbd2_journal_try_to_free_buffers() kernel-doc comment
      354c6e07
    • Linus Torvalds's avatar
      Merge tag '5.19-rc2-smb3-client-fixes' of git://git.samba.org/sfrench/cifs-2.6 · ace2045e
      Linus Torvalds authored
      Pull cifs client fixes from Steve French:
       "Two cifs debugging improvements - one found to deal with debugging a
        multichannel problem and one for a recent fallocate issue
      
        This does include the two larger multichannel reconnect (dynamically
        adjusting interfaces on reconnect) patches, because we recently found
        an additional problem with multichannel to one server type that I want
        to include at the same time"
      
      * tag '5.19-rc2-smb3-client-fixes' of git://git.samba.org/sfrench/cifs-2.6:
        cifs: when a channel is not found for server, log its connection id
        smb3: add trace point for SMB2_set_eof
      ace2045e
  4. 18 Jun, 2022 1 commit