1. 30 Aug, 2018 34 commits
    • Paul E. McKenney's avatar
      rcu: Remove rsp parameter from rcu_gp_kthread_wake() · 532c00c9
      Paul E. McKenney authored
      There now is only one rcu_state structure in a given build of the
      Linux kernel, so there is no need to pass it as a parameter to RCU's
      functions.  This commit therefore removes the rsp parameter from
      rcu_gp_kthread_wake().
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      532c00c9
    • Paul E. McKenney's avatar
      rcu: Remove rsp parameter from rcu_future_gp_cleanup() · 3481f2ea
      Paul E. McKenney authored
      There now is only one rcu_state structure in a given build of the
      Linux kernel, so there is no need to pass it as a parameter to RCU's
      functions.  This commit therefore removes the rsp parameter from
      rcu_future_gp_cleanup().
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      3481f2ea
    • Paul E. McKenney's avatar
      rcu: Remove rsp parameter from check_cpu_stall() · ea12ff2b
      Paul E. McKenney authored
      There now is only one rcu_state structure in a given build of the
      Linux kernel, so there is no need to pass it as a parameter to RCU's
      functions.  This commit therefore removes the rsp parameter from
      check_cpu_stall().
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      ea12ff2b
    • Paul E. McKenney's avatar
      rcu: Remove rsp parameter from print_cpu_stall() · 4e8b8e08
      Paul E. McKenney authored
      There now is only one rcu_state structure in a given build of the
      Linux kernel, so there is no need to pass it as a parameter to RCU's
      functions.  This commit therefore removes the rsp parameter from
      print_cpu_stall().
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      4e8b8e08
    • Paul E. McKenney's avatar
      rcu: Remove rsp parameter from print_other_cpu_stall() · a91e7e58
      Paul E. McKenney authored
      There now is only one rcu_state structure in a given build of the
      Linux kernel, so there is no need to pass it as a parameter to RCU's
      functions.  This commit therefore removes the rsp parameter from
      print_other_cpu_stall().
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      a91e7e58
    • Paul E. McKenney's avatar
      rcu: Remove rsp parameter from rcu_stall_kick_kthreads() · e1741c69
      Paul E. McKenney authored
      There now is only one rcu_state structure in a given build of the
      Linux kernel, so there is no need to pass it as a parameter to RCU's
      functions.  This commit therefore removes the rsp parameter from
      rcu_stall_kick_kthreads().
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      e1741c69
    • Paul E. McKenney's avatar
      rcu: Remove rsp parameter from rcu_dump_cpu_stacks() · 33dbdbf0
      Paul E. McKenney authored
      There now is only one rcu_state structure in a given build of the
      Linux kernel, so there is no need to pass it as a parameter to RCU's
      functions.  This commit therefore removes the rsp parameter from
      rcu_dump_cpu_stacks().
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      33dbdbf0
    • Paul E. McKenney's avatar
      rcu: Remove rsp parameter from rcu_check_gp_kthread_starvation() · 8fd119b6
      Paul E. McKenney authored
      There now is only one rcu_state structure in a given build of the
      Linux kernel, so there is no need to pass it as a parameter to RCU's
      functions.  This commit therefore removes the rsp parameter from
      rcu_check_gp_kthread_starvation().
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      8fd119b6
    • Paul E. McKenney's avatar
      rcu: Remove rsp parameter from record_gp_stall_check_time() · ad3832e9
      Paul E. McKenney authored
      There now is only one rcu_state structure in a given build of the
      Linux kernel, so there is no need to pass it as a parameter to RCU's
      functions.  This commit therefore removes the rsp parameter from
      record_gp_stall_check_time().
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      ad3832e9
    • Paul E. McKenney's avatar
      rcu: Remove rsp parameter from rcu_get_root() · 336a4f6c
      Paul E. McKenney authored
      There now is only one rcu_state structure in a given build of the
      Linux kernel, so there is no need to pass it as a parameter to RCU's
      functions.  This commit therefore removes the rsp parameter from
      rcu_get_root().
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      336a4f6c
    • Paul E. McKenney's avatar
      rcu: Remove rsp parameter from rcu_gp_in_progress() · de8e8730
      Paul E. McKenney authored
      There now is only one rcu_state structure in a given build of the
      Linux kernel, so there is no need to pass it as a parameter to RCU's
      functions.  This commit therefore removes the rsp parameter from
      rcu_gp_in_progress().
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      de8e8730
    • Paul E. McKenney's avatar
      rcu: Remove rsp parameter from rcu_report_qs_rdp() · 33085c46
      Paul E. McKenney authored
      There now is only one rcu_state structure in a given build of the
      Linux kernel, so there is no need to pass it as a parameter to RCU's
      functions.  This commit therefore removes the rsp parameter from
      rcu_report_qs_rdp().
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      33085c46
    • Paul E. McKenney's avatar
      rcu: Remove rsp parameter from rcu_report_unblock_qs_rnp() · 139ad4da
      Paul E. McKenney authored
      There now is only one rcu_state structure in a given build of the
      Linux kernel, so there is no need to pass it as a parameter to RCU's
      functions.  This commit therefore removes the rsp parameter from
      rcu_report_unblock_qs_rnp(), which is particularly appropriate in
      this case given that this parameter is no longer used.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      139ad4da
    • Paul E. McKenney's avatar
      rcu: Remove rsp parameter from rcu_report_qs_rsp() · aff4e9ed
      Paul E. McKenney authored
      There now is only one rcu_state structure in a given build of the
      Linux kernel, so there is no need to pass it as a parameter to RCU's
      functions.  This commit therefore removes the rsp parameter from
      rcu_report_qs_rsp().
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      aff4e9ed
    • Paul E. McKenney's avatar
      rcu: Remove rsp parameter from rcu_report_qs_rnp() · b50912d0
      Paul E. McKenney authored
      There now is only one rcu_state structure in a given build of the
      Linux kernel, so there is no need to pass it as a parameter to RCU's
      functions.  This commit therefore removes the rsp parameter from
      rcu_report_qs_rnp().
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      b50912d0
    • Paul E. McKenney's avatar
      rcu: Remove rcu_data_p pointer to default rcu_data structure · 2280ee5a
      Paul E. McKenney authored
      The rcu_data_p pointer references the default set of per-CPU rcu_data
      structures, that is, those that call_rcu() uses, as opposed to
      call_rcu_bh() and sometimes call_rcu_sched().  But there is now only one
      set of per-CPU rcu_data structures, so that one set is by definition
      the default, which means that the rcu_data_p pointer no longer serves
      any useful purpose.  This commit therefore removes it.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      2280ee5a
    • Paul E. McKenney's avatar
      rcu: Remove rcu_state_p pointer to default rcu_state structure · 16fc9c60
      Paul E. McKenney authored
      The rcu_state_p pointer references the default rcu_state structure,
      that is, the one that call_rcu() uses, as opposed to call_rcu_bh()
      and sometimes call_rcu_sched().  But there is now only one rcu_state
      structure, so that one structure is by definition the default, which
      means that the rcu_state_p pointer no longer serves any useful purpose.
      This commit therefore removes it.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      16fc9c60
    • Paul E. McKenney's avatar
      rcu: Remove rcu_state structure's ->rda field · da1df50d
      Paul E. McKenney authored
      The rcu_state structure's ->rda field was used to find the per-CPU
      rcu_data structures corresponding to that rcu_state structure.  But now
      there is only one rcu_state structure (creatively named "rcu_state")
      and one set of per-CPU rcu_data structures (creatively named "rcu_data").
      Therefore, uses of the ->rda field can always be replaced by "rcu_data,
      and this commit makes that change and removes the ->rda field.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      da1df50d
    • Paul E. McKenney's avatar
      rcu: Eliminate rcu_state structure's ->call field · ec5dd444
      Paul E. McKenney authored
      The rcu_state structure's ->call field references the corresponding RCU
      flavor's call_rcu() function.  However, now that there is only ever one
      rcu_state structure in a given build of the Linux kernel, and that flavor
      uses plain old call_rcu(), there is not a lot of point in continuing to
      have the ->call field.  This commit therefore removes it.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      ec5dd444
    • Paul E. McKenney's avatar
      rcu: Remove RCU_STATE_INITIALIZER() · 358be2d3
      Paul E. McKenney authored
      Now that a given build of the Linux kernel has only one set of rcu_state,
      rcu_node, and rcu_data structures, there is no point in creating a macro
      to declare and compile-time initialize them.  This commit therefore
      just does normal declaration and compile-time initialization of these
      structures.  While in the area, this commit also removes #ifndefs of
      the no-longer-ever-defined preprocessor macro RCU_TREE_NONCORE.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      358be2d3
    • Paul E. McKenney's avatar
      rcu: Express Tiny RCU updates in terms of RCU rather than RCU-sched · 709fdce7
      Paul E. McKenney authored
      This commit renames Tiny RCU functions so that the lowest level of
      functionality is RCU (e.g., synchronize_rcu()) rather than RCU-sched
      (e.g., synchronize_sched()).  This provides greater naming compatibility
      with Tree RCU, which will in turn permit more LoC removal once
      the RCU-sched and RCU-bh update-side API is removed.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      [ paulmck: Fix Tiny call_rcu()'s EXPORT_SYMBOL() in response to a bug
        report from kbuild test robot. ]
      709fdce7
    • Paul E. McKenney's avatar
      rcu: Define RCU-sched API in terms of RCU for Tree RCU PREEMPT builds · 45975c7d
      Paul E. McKenney authored
      Now that RCU-preempt knows about preemption disabling, its implementation
      of synchronize_rcu() works for synchronize_sched(), and likewise for the
      other RCU-sched update-side API members.  This commit therefore confines
      the RCU-sched update-side code to CONFIG_PREEMPT=n builds, and defines
      RCU-sched's update-side API members in terms of those of RCU-preempt.
      
      This means that any given build of the Linux kernel has only one
      update-side flavor of RCU, namely RCU-preempt for CONFIG_PREEMPT=y builds
      and RCU-sched for CONFIG_PREEMPT=n builds.  This in turn means that kernels
      built with CONFIG_RCU_NOCB_CPU=y have only one rcuo kthread per CPU.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: Andi Kleen <ak@linux.intel.com>
      45975c7d
    • Paul E. McKenney's avatar
    • Paul E. McKenney's avatar
      rcu: Drop "wake" parameter from rcu_report_exp_rdp() · 2bbfc25b
      Paul E. McKenney authored
      The rcu_report_exp_rdp() function is always invoked with its "wake"
      argument set to "true", so this commit drops this parameter.  The only
      potential call site that would use "false" is in the code driving the
      expedited grace period, and that code uses rcu_report_exp_cpu_mult()
      instead, which therefore retains its "wake" parameter.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      2bbfc25b
    • Paul E. McKenney's avatar
      rcu: Update comments and help text for no more RCU-bh updaters · 82fcecfa
      Paul E. McKenney authored
      This commit updates comments and help text to account for the fact that
      RCU-bh update-side functions are now simple wrappers for their RCU or
      RCU-sched counterparts.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      82fcecfa
    • Paul E. McKenney's avatar
      rcu: Define RCU-bh update API in terms of RCU · 65cfe358
      Paul E. McKenney authored
      Now that the main RCU API knows about softirq disabling and softirq's
      quiescent states, the RCU-bh update code can be dispensed with.
      This commit therefore removes the RCU-bh update-side implementation and
      defines RCU-bh's update-side API in terms of that of either RCU-preempt or
      RCU-sched, depending on the setting of the CONFIG_PREEMPT Kconfig option.
      
      In kernels built with CONFIG_RCU_NOCB_CPU=y this has the knock-on effect
      of reducing by one the number of rcuo kthreads per CPU.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      65cfe358
    • Paul E. McKenney's avatar
      rcu: Report expedited grace periods at context-switch time · ba1c64c2
      Paul E. McKenney authored
      This commit reduces the latency of expedited RCU grace periods by
      reporting a quiescent state for the CPU at context-switch time.
      In CONFIG_PREEMPT=y kernels, if the outgoing task is still within an
      RCU read-side critical section (and thus still blocking some grace
      period, perhaps including this expedited grace period), then that task
      will already have been placed on one of the leaf rcu_node structures'
      ->blkd_tasks list.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      ba1c64c2
    • Paul E. McKenney's avatar
      rcu: Apply RCU-bh QSes to RCU-sched and RCU-preempt when safe · d28139c4
      Paul E. McKenney authored
      One necessary step towards consolidating the three flavors of RCU is to
      make sure that the resulting consolidated "one flavor to rule them all"
      correctly handles networking denial-of-service attacks.  One thing that
      allows RCU-bh to do so is that __do_softirq() invokes rcu_bh_qs() every
      so often, and so something similar has to happen for consolidated RCU.
      
      This must be done carefully.  For example, if a preemption-disabled
      region of code takes an interrupt which does softirq processing before
      returning, consolidated RCU must ignore the resulting rcu_bh_qs()
      invocations -- preemption is still disabled, and that means an RCU
      reader for the consolidated flavor.
      
      This commit therefore creates a new rcu_softirq_qs() that is called only
      from the ksoftirqd task, thus avoiding the interrupted-a-preempted-region
      problem.  This new rcu_softirq_qs() function invokes rcu_sched_qs(),
      rcu_preempt_qs(), and rcu_preempt_deferred_qs().  The latter call handles
      any deferred quiescent states.
      
      Note that __do_softirq() still invokes rcu_bh_qs().  It will continue to
      do so until a later stage of cleanup when the RCU-bh flavor is removed.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      [ paulmck: Fix !SMP issue located by kbuild test robot. ]
      d28139c4
    • Paul E. McKenney's avatar
      rcu: Add warning to detect half-interrupts · e11ec65c
      Paul E. McKenney authored
      RCU's dyntick-idle code is written to tolerate half-interrupts, that it,
      either an interrupt that invokes rcu_irq_enter() but never invokes the
      corresponding rcu_irq_exit() on the one hand, or an interrupt that never
      invokes rcu_irq_enter() but does invoke the "corresponding" rcu_irq_exit()
      on the other.  These things really did happen at one time, as evidenced
      by this ca-2011 LKML post:
      
      http://lkml.kernel.org/r/20111014170019.GE2428@linux.vnet.ibm.com
      
      The reason why RCU tolerates half-interrupts is that usermode helpers
      used exceptions to invoke a system call from within the kernel such that
      the system call did a normal return (not a return from exception) to
      the calling context.  This caused rcu_irq_enter() to be invoked without
      a matching rcu_irq_exit().  However, usermode helpers have since been
      rewritten to make much more housebroken use of workqueues, kernel threads,
      and do_execve(), and therefore should no longer produce half-interrupts.
      No one knows of any other source of half-interrupts, but then again,
      no one seems insane enough to go audit the entire kernel to verify that
      half-interrupts really are a relic of the past.
      
      This commit therefore adds a pair of WARN_ON_ONCE() calls that will
      trigger in the presence of half interrupts, which the code will continue
      to handle correctly.  If neither of these WARN_ON_ONCE() trigger by
      mid-2021, then perhaps RCU can stop handling half-interrupts, which
      would be a considerable simplification.
      Reported-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      Reported-by: default avatarJoel Fernandes <joel@joelfernandes.org>
      Reported-by: default avatarAndy Lutomirski <luto@kernel.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Reviewed-by: default avatarJoel Fernandes (Google) <joel@joelfernandes.org>
      e11ec65c
    • Paul E. McKenney's avatar
      rcu: Remove now-unused ->b.exp_need_qs field from the rcu_special union · fcc878e4
      Paul E. McKenney authored
      The ->b.exp_need_qs field is now set only to false, so this commit
      removes it.  The job this field used to do is now done by the rcu_data
      structure's ->deferred_qs field, which is a consequence of a better
      split between task-based (the rcu_node structure's ->exp_tasks field) and
      CPU-based (the aforementioned rcu_data structure's ->deferred_qs field)
      tracking of quiescent states for RCU-preempt expedited grace periods.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      fcc878e4
    • Paul E. McKenney's avatar
      rcu: Allow processing deferred QSes for exiting RCU-preempt readers · 27c744e3
      Paul E. McKenney authored
      If an RCU-preempt read-side critical section is exiting, that is,
      ->rcu_read_lock_nesting is negative, then it is a good time to look
      at the possibility of reporting deferred quiescent states.  This
      commit therefore updates the checks in rcu_preempt_need_deferred_qs()
      to allow exiting critical sections to report deferred quiescent states.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      27c744e3
    • Paul E. McKenney's avatar
      rcutorture: Test extended "rcu" read-side critical sections · c0335743
      Paul E. McKenney authored
      This commit makes the "rcu" torture type test extended read-side
      critical sections in order to test the deferral of RCU-preempt
      quiescent-state testing.
      
      In CONFIG_PREEMPT=n kernels, this simply duplicates the setup already
      in place for the "sched" torture type.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      c0335743
    • Paul E. McKenney's avatar
      rcu: Defer reporting RCU-preempt quiescent states when disabled · 3e310098
      Paul E. McKenney authored
      This commit defers reporting of RCU-preempt quiescent states at
      rcu_read_unlock_special() time when any of interrupts, softirq, or
      preemption are disabled.  These deferred quiescent states are reported
      at a later RCU_SOFTIRQ, context switch, idle entry, or CPU-hotplug
      offline operation.  Of course, if another RCU read-side critical
      section has started in the meantime, the reporting of the quiescent
      state will be further deferred.
      
      This also means that disabling preemption, interrupts, and/or
      softirqs will act as an RCU-preempt read-side critical section.
      This is enforced by checking preempt_count() as needed.
      
      Some special cases must be handled on an ad-hoc basis, for example,
      context switch is a quiescent state even though both the scheduler and
      do_exit() disable preemption.  In these cases, additional calls to
      rcu_preempt_deferred_qs() override the preemption disabling.  Similar
      logic overrides disabled interrupts in rcu_preempt_check_callbacks()
      because in this case the quiescent state happened just before the
      corresponding scheduling-clock interrupt.
      
      In theory, this change lifts a long-standing restriction that required
      that if interrupts were disabled across a call to rcu_read_unlock()
      that the matching rcu_read_lock() also be contained within that
      interrupts-disabled region of code.  Because the reporting of the
      corresponding RCU-preempt quiescent state is now deferred until
      after interrupts have been enabled, it is no longer possible for this
      situation to result in deadlocks involving the scheduler's runqueue and
      priority-inheritance locks.  This may allow some code simplification that
      might reduce interrupt latency a bit.  Unfortunately, in practice this
      would also defer deboosting a low-priority task that had been subjected
      to RCU priority boosting, so real-time-response considerations might
      well force this restriction to remain in place.
      
      Because RCU-preempt grace periods are now blocked not only by RCU
      read-side critical sections, but also by disabling of interrupts,
      preemption, and softirqs, it will be possible to eliminate RCU-bh and
      RCU-sched in favor of RCU-preempt in CONFIG_PREEMPT=y kernels.  This may
      require some additional plumbing to provide the network denial-of-service
      guarantees that have been traditionally provided by RCU-bh.  Once these
      are in place, CONFIG_PREEMPT=n kernels will be able to fold RCU-bh
      into RCU-sched.  This would mean that all kernels would have but
      one flavor of RCU, which would open the door to significant code
      cleanup.
      
      Moving to a single flavor of RCU would also have the beneficial effect
      of reducing the NOCB kthreads by at least a factor of two.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      [ paulmck: Apply rcu_read_unlock_special() preempt_count() feedback
        from Joel Fernandes. ]
      [ paulmck: Adjust rcu_eqs_enter() call to rcu_preempt_deferred_qs() in
        response to bug reports from kbuild test robot. ]
      [ paulmck: Fix bug located by kbuild test robot involving recursion
        via rcu_preempt_deferred_qs(). ]
      3e310098
    • Byungchul Park's avatar
      rcu: Refactor rcu_{nmi,irq}_{enter,exit}() · cf7614e1
      Byungchul Park authored
      When entering or exiting irq or NMI handlers, the current code uses
      ->dynticks_nmi_nesting to detect if it is in the outermost handler,
      that is, the one interrupting or returning to an RCU-idle context (the
      idle loop or nohz_full usermode execution).  When entering the outermost
      handler via an interrupt (as opposed to NMI), it is necessary to invoke
      rcu_dynticks_task_exit() just before the CPU is marked non-idle from an
      RCU perspective and to invoke rcu_cleanup_after_idle() just after the
      CPU is marked non-idle.  Similarly, when exiting the outermost handler
      via an interrupt, it is necessary to invoke rcu_prepare_for_idle() just
      before marking the CPU idle and to invoke rcu_dynticks_task_enter()
      just after marking the CPU idle.
      
      The decision to execute these four functions is currently taken in
      rcu_irq_enter() and rcu_irq_exit() as follows:
      
         rcu_irq_enter()
            /* A conditional branch with ->dynticks_nmi_nesting */
            rcu_nmi_enter()
               /* A conditional branch with ->dynticks */
            /* A conditional branch with ->dynticks_nmi_nesting */
      
         rcu_irq_exit()
            /* A conditional branch with ->dynticks_nmi_nesting */
            rcu_nmi_exit()
               /* A conditional branch with ->dynticks_nmi_nesting */
            /* A conditional branch with ->dynticks_nmi_nesting */
      
         rcu_nmi_enter()
            /* A conditional branch with ->dynticks */
      
         rcu_nmi_exit()
            /* A conditional branch with ->dynticks_nmi_nesting */
      
      This works, but the conditional branches in rcu_irq_enter() and
      rcu_irq_exit() are redundant with those in rcu_nmi_enter() and
      rcu_nmi_exit(), respectively.  Redundant branches are not something
      we want in the to/from-idle fastpaths, so this commit refactors
      rcu_{nmi,irq}_{enter,exit}() so they use a common inlined function passed
      a constant argument as follows:
      
         rcu_irq_enter() inlining rcu_nmi_enter_common(irq=true)
            /* A conditional branch with ->dynticks */
      
         rcu_irq_exit() inlining rcu_nmi_exit_common(irq=true)
            /* A conditional branch with ->dynticks_nmi_nesting */
      
         rcu_nmi_enter() inlining rcu_nmi_enter_common(irq=false)
            /* A conditional branch with ->dynticks */
      
         rcu_nmi_exit() inlining rcu_nmi_exit_common(irq=false)
            /* A conditional branch with ->dynticks_nmi_nesting */
      
      The combination of the constant function argument and the inlining allows
      the compiler to discard the conditionals that previously controlled
      execution of rcu_dynticks_task_exit(), rcu_cleanup_after_idle(),
      rcu_prepare_for_idle(), and rcu_dynticks_task_enter().  This reduces both
      the to-idle and from-idle path lengths by two conditional branches each,
      and improves readability as well.
      
      This commit also changes order of execution from this:
      
      	rcu_dynticks_task_exit();
      	rcu_dynticks_eqs_exit();
      	trace_rcu_dyntick();
      	rcu_cleanup_after_idle();
      
      To this:
      
      	rcu_dynticks_task_exit();
      	rcu_dynticks_eqs_exit();
      	rcu_cleanup_after_idle();
      	trace_rcu_dyntick();
      
      In other words, the calls to rcu_cleanup_after_idle() and
      trace_rcu_dyntick() are reversed.  This has no functional effect because
      the real concern is whether a given call is before or after the call to
      rcu_dynticks_eqs_exit(), and this patch does not change that.  Before the
      call to rcu_dynticks_eqs_exit(), RCU is not yet watching the current
      CPU and after that call RCU is watching.
      
      A similar switch in calling order happens on the idle-entry path, with
      similar lack of effect for the same reasons.
      Suggested-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Signed-off-by: default avatarByungchul Park <byungchul.park@lge.com>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      [ paulmck: Applied Steven Rostedt feedback. ]
      Reviewed-by: default avatarSteven Rostedt (VMware) <rostedt@goodmis.org>
      cf7614e1
  2. 26 Aug, 2018 6 commits
    • Linus Torvalds's avatar
      Linux 4.19-rc1 · 5b394b2d
      Linus Torvalds authored
      5b394b2d
    • Linus Torvalds's avatar
      Merge branch 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · b933d6eb
      Linus Torvalds authored
      Pull timer update from Thomas Gleixner:
       "New defines for the compat time* types so they can be shared between
        32bit and 64bit builds. Not used yet, but merging them now allows the
        actual conversions to be merged through different maintainer trees
        without dependencies
      
        We still have compat interfaces for 32bit on 64bit even with the new
        2038 safe timespec/val variants because pointer size is different. And
        for the old style timespec/val interfaces we need yet another 'compat'
        interface for both 32bit native and 32bit on 64bit"
      
      * 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
        y2038: Provide aliases for compat helpers
      b933d6eb
    • Linus Torvalds's avatar
      Merge branch 'ida-4.19' of git://git.infradead.org/users/willy/linux-dax · aba16dc5
      Linus Torvalds authored
      Pull IDA updates from Matthew Wilcox:
       "A better IDA API:
      
            id = ida_alloc(ida, GFP_xxx);
            ida_free(ida, id);
      
        rather than the cumbersome ida_simple_get(), ida_simple_remove().
      
        The new IDA API is similar to ida_simple_get() but better named.  The
        internal restructuring of the IDA code removes the bitmap
        preallocation nonsense.
      
        I hope the net -200 lines of code is convincing"
      
      * 'ida-4.19' of git://git.infradead.org/users/willy/linux-dax: (29 commits)
        ida: Change ida_get_new_above to return the id
        ida: Remove old API
        test_ida: check_ida_destroy and check_ida_alloc
        test_ida: Convert check_ida_conv to new API
        test_ida: Move ida_check_max
        test_ida: Move ida_check_leaf
        idr-test: Convert ida_check_nomem to new API
        ida: Start new test_ida module
        target/iscsi: Allocate session IDs from an IDA
        iscsi target: fix session creation failure handling
        drm/vmwgfx: Convert to new IDA API
        dmaengine: Convert to new IDA API
        ppc: Convert vas ID allocation to new IDA API
        media: Convert entity ID allocation to new IDA API
        ppc: Convert mmu context allocation to new IDA API
        Convert net_namespace to new IDA API
        cb710: Convert to new IDA API
        rsxx: Convert to new IDA API
        osd: Convert to new IDA API
        sd: Convert to new IDA API
        ...
      aba16dc5
    • Linus Torvalds's avatar
      Merge tag 'gcc-plugins-v4.19-rc1-fix' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux · c4726e77
      Linus Torvalds authored
      Pull gcc plugin fix from Kees Cook:
       "Lift gcc test into Kconfig. This is for better behavior when the
        kernel is built with Clang, reported by Stefan Agner"
      
      * tag 'gcc-plugins-v4.19-rc1-fix' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux:
        gcc-plugins: Disable when building under Clang
      c4726e77
    • Linus Torvalds's avatar
      Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · d207ea8e
      Linus Torvalds authored
      Pull perf updates from Thomas Gleixner:
       "Kernel:
         - Improve kallsyms coverage
         - Add x86 entry trampolines to kcore
         - Fix ARM SPE handling
         - Correct PPC event post processing
      
        Tools:
         - Make the build system more robust
         - Small fixes and enhancements all over the place
         - Update kernel ABI header copies
         - Preparatory work for converting libtraceevnt to a shared library
         - License cleanups"
      
      * 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (100 commits)
        tools arch: Update arch/x86/lib/memcpy_64.S copy used in 'perf bench mem memcpy'
        tools arch x86: Update tools's copy of cpufeatures.h
        perf python: Fix pyrf_evlist__read_on_cpu() interface
        perf mmap: Store real cpu number in 'struct perf_mmap'
        perf tools: Remove ext from struct kmod_path
        perf tools: Add gzip_is_compressed function
        perf tools: Add lzma_is_compressed function
        perf tools: Add is_compressed callback to compressions array
        perf tools: Move the temp file processing into decompress_kmodule
        perf tools: Use compression id in decompress_kmodule()
        perf tools: Store compression id into struct dso
        perf tools: Add compression id into 'struct kmod_path'
        perf tools: Make is_supported_compression() static
        perf tools: Make decompress_to_file() function static
        perf tools: Get rid of dso__needs_decompress() call in __open_dso()
        perf tools: Get rid of dso__needs_decompress() call in symbol__disassemble()
        perf tools: Get rid of dso__needs_decompress() call in read_object_code()
        tools lib traceevent: Change to SPDX License format
        perf llvm: Allow passing options to llc in addition to clang
        perf parser: Improve error message for PMU address filters
        ...
      d207ea8e
    • Linus Torvalds's avatar
      Merge branch 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · 2a8a2b7c
      Linus Torvalds authored
      Pull x86 fixes from Thomas Gleixner:
      
       - Correct the L1TF fallout on 32bit and the off by one in the 'too much
         RAM for protection' calculation.
      
       - Add a helpful kernel message for the 'too much RAM' case
      
       - Unbreak the VDSO in case that the compiler desides to use indirect
         jumps/calls and emits retpolines which cannot be resolved because the
         kernel uses its own thunks, which does not work for the VDSO. Make it
         use the builtin thunks.
      
       - Re-export start_thread() which was unexported when the 32/64bit
         implementation was unified. start_thread() is required by modular
         binfmt handlers.
      
       - Trivial cleanups
      
      * 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
        x86/speculation/l1tf: Suggest what to do on systems with too much RAM
        x86/speculation/l1tf: Fix off-by-one error when warning that system has too much RAM
        x86/kvm/vmx: Remove duplicate l1d flush definitions
        x86/speculation/l1tf: Fix overflow in l1tf_pfn_limit() on 32bit
        x86/process: Re-export start_thread()
        x86/mce: Add notifier_block forward declaration
        x86/vdso: Fix vDSO build if a retpoline is emitted
      2a8a2b7c