1. 19 Jun, 2019 1 commit
    • Paul E. McKenney's avatar
      Merge branches 'consolidate.2019.05.28a', 'doc.2019.05.28a',... · 11ca7a9d
      Paul E. McKenney authored
      Merge branches 'consolidate.2019.05.28a', 'doc.2019.05.28a', 'fixes.2019.06.13a', 'srcu.2019.05.28a', 'sync.2019.05.28a' and 'torture.2019.05.28a' into HEAD
      
      consolidate.2019.05.28a: RCU flavor consolidation cleanups and optmizations.
      doc.2019.05.28a: Documentation updates.
      fixes.2019.06.13a: Miscellaneous fixes.
      srcu.2019.05.28a: SRCU updates.
      sync.2019.05.28a: RCU-sync flavor consolidation.
      torture.2019.05.28a: Torture-test updates.
      11ca7a9d
  2. 13 Jun, 2019 4 commits
    • Andrea Parri's avatar
      rcu: Don't return a value from rcu_assign_pointer() · 9129b017
      Andrea Parri authored
      Quoting Paul [1]:
      
        "Given that a quick (and perhaps error-prone) search of the uses
         of rcu_assign_pointer() in v5.1 didn't find a single use of the
         return value, let's please instead change the documentation and
         implementation to eliminate the return value."
      
      [1] https://lkml.kernel.org/r/20190523135013.GL28207@linux.ibm.comSigned-off-by: default avatarAndrea Parri <andrea.parri@amarulasolutions.com>
      Cc: "Paul E. McKenney" <paulmck@linux.ibm.com>
      Cc: Josh Triplett <josh@joshtriplett.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
      Cc: Lai Jiangshan <jiangshanlai@gmail.com>
      Cc: Joel Fernandes <joel@joelfernandes.org>
      Cc: rcu@vger.kernel.org
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Will Deacon <will.deacon@arm.com>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Cc: Matthew Wilcox <willy@infradead.org>
      Cc: Sasha Levin <sashal@kernel.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      9129b017
    • Waiman Long's avatar
      rcu: Force inlining of rcu_read_lock() · 6da9f775
      Waiman Long authored
      When debugging options are turned on, the rcu_read_lock() function
      might not be inlined. This results in lockdep's print_lock() function
      printing "rcu_read_lock+0x0/0x70" instead of rcu_read_lock()'s caller.
      For example:
      
      [   10.579995] =============================
      [   10.584033] WARNING: suspicious RCU usage
      [   10.588074] 4.18.0.memcg_v2+ #1 Not tainted
      [   10.593162] -----------------------------
      [   10.597203] include/linux/rcupdate.h:281 Illegal context switch in
      RCU read-side critical section!
      [   10.606220]
      [   10.606220] other info that might help us debug this:
      [   10.606220]
      [   10.614280]
      [   10.614280] rcu_scheduler_active = 2, debug_locks = 1
      [   10.620853] 3 locks held by systemd/1:
      [   10.624632]  #0: (____ptrval____) (&type->i_mutex_dir_key#5){.+.+}, at: lookup_slow+0x42/0x70
      [   10.633232]  #1: (____ptrval____) (rcu_read_lock){....}, at: rcu_read_lock+0x0/0x70
      [   10.640954]  #2: (____ptrval____) (rcu_read_lock){....}, at: rcu_read_lock+0x0/0x70
      
      These "rcu_read_lock+0x0/0x70" strings are not providing any useful
      information.  This commit therefore forces inlining of the rcu_read_lock()
      function so that rcu_read_lock()'s caller is instead shown.
      Signed-off-by: default avatarWaiman Long <longman@redhat.com>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      6da9f775
    • Paul E. McKenney's avatar
    • Paul E. McKenney's avatar
      rcu: Upgrade sync_exp_work_done() to smp_mb() · 96050c68
      Paul E. McKenney authored
      The sync_exp_work_done() function uses smp_mb__before_atomic(), but
      there is no obvious atomic in the ensuing code.  The ordering is
      absolutely required for grace periods to work correctly, so this
      commit upgrades the smp_mb__before_atomic() to smp_mb().
      
      Fixes: 6fba2b37 ("rcu: Remove deprecated RCU debugfs tracing code")
      Reported-by: default avatarAndrea Parri <andrea.parri@amarulasolutions.com>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      96050c68
  3. 28 May, 2019 35 commits
    • Paul E. McKenney's avatar
      rcutorture: Upper case solves the case of the vanishing NULL pointer · 354ea05d
      Paul E. McKenney authored
      Various security techniques can obfuscate pointer printouts on the
      console.  Unfortunately, rcutorture relies on either "null" or all zeroes
      to identify the last few statistics printouts at the end of the test.
      These need to be identified because failing to do so will results in
      false-positive complaints about grace-period hangs.
      
      This commit therefore prints the "ver:" in capitals ("VER:") when
      the RCU-protected pointer has been set to NULL, which causes rcutorture's
      parse-console.sh script to correctly ignore these lines.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      354ea05d
    • Paul E. McKenney's avatar
      torture: Suppress propagating trace_printk() warning · 8997e631
      Paul E. McKenney authored
      When trace_printk() is used, a message including "BUG" is printed to
      the console, which fools the rcutorture scripting into believing that
      the corresponding test scenario failed.  This commit therefore filters
      out this particular instance of "BUG", thus avoiding the false-positive
      test-failure report.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      8997e631
    • Paul E. McKenney's avatar
    • Paul E. McKenney's avatar
      torture: Add --trust-make to suppress "make clean" · b93c765f
      Paul E. McKenney authored
      The current rcutorture scripts unconditionally do "make clean", which is
      a good way of getting the needed testing done despite any imperfections in
      Makefile dependency tracking.  However, this can be a bit irritating when
      repeatedly running a single scenario after small changes, for example,
      when debugging a problem that affects only a single scenario.  This commit
      therefore adds a --trust-make argument that suppresses the "make clean".
      
      Even when using ccache, this speeds up kernel builds by up to almost an
      order of magnitude on my laptop.
      Reported-by: default avatarPeter Zijlstra <peterz@infradead.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      b93c765f
    • Paul E. McKenney's avatar
      torture: Make --cpus override idleness calculations · 7225c077
      Paul E. McKenney authored
      Currently, rcutorture will use relatively few CPUs to build the kernel
      on a busy system, which is often as it should be.  However, if the user
      has used the --cpus argument to dedicate a specified number of CPUs to
      this torture test, it would be good if the kernel build also made use
      of them.  This commit therefore changes the cpus2use.sh script to use
      --cpus when specified and to do the idleness calculations otherwise.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      7225c077
    • Paul E. McKenney's avatar
      torture: Run kernel build in source directory · 6dc82595
      Paul E. McKenney authored
      For historical reasons, rcutorture places its build products in a
      tools/testing/selftests/rcutorture/b1 directory using the O= kbuild
      command-line argument.  However, doing this requires that the source
      directory be pristine: Not just "make clean" pristine, but instead "make
      mrproper" (or, equivalently, "make distclean") pristine.  Therefore,
      rcutorture executes a "make mrproper" before each build.  Unfortunately,
      "make mrproper" has the side effect of removing pretty much everything,
      including tags files and cscope databases, which can be inconvenient
      to people whose workflow centers around a single source tree.
      
      This commit therefore makes rcutorture do the build directly in the
      source directory, removing the need for "make mrproper".  This works
      because all needed build products are moved to their proper place in the
      "res" directory immediately after the build completes, so that multiple
      rcutorture kernels can still run concurrently.
      Reported-by: default avatarPeter Zijlstra <peterz@infradead.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      6dc82595
    • Paul E. McKenney's avatar
    • Paul E. McKenney's avatar
      torture: Capture qemu output · 7dedfd43
      Paul E. McKenney authored
      Currently qemu output appears on standard output, but is inaccessible
      later on.  This commit therefore captures this output and causes
      kvm-recheck.sh to output this output if QEMU gave a non-zero non-137
      exit code.  (And exit code of 137 indicates that QEMU was killed, in
      which case we want to know about the hang rather than the fact that
      QEMU was killed.)
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      7dedfd43
    • Sebastian Andrzej Siewior's avatar
      rcutorture: Tweak kvm options · a6fda6da
      Sebastian Andrzej Siewior authored
      In one of my rcutorture tests the TSC clocksource got marked unstable
      due to a large difference in the TSC value. I'm not sure if the guest
      run for a long time with disabled interrupts or if the host was very
      busy and didn't schedule the guest for some time.
      
      I took a look on the qemu/KVM options and decided to update the options:
      
      - Use kvm{32|64} as CPU. We could probably use `host' (like ARM does)
        for maximum available features but since we don't run any userland I'm
        not sure if it makes any difference.
      
      - Drop the "noapic" option. There is no history why the APIC was disabled,
        I see no reason for it.  Once old qemu versions fade away, we can add
        "x2apic=on,tsc-deadline=on,hypervisor=on,tsc_adjust=on".
      
      - Additional config options. It ensures that the kernel knowns that it
        runs as a kvm guest and can use virt devices like the kvm-clock as
        clocksource. The kvm-clock was the main motivation here.
      
      - I didn't add a random HW device. It would make the random device ready
        earlier (not it doesn't complete the initialisation at all) but I
        doubt that there is any need for this.
      Signed-off-by: default avatarSebastian Andrzej Siewior <bigeasy@linutronix.de>
      [ paulmck: The world is not quite ready for CONFIG_PARAVIRT_SPINLOCKS=y
        and x2apic, so they are omitted for the time being. ]
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      a6fda6da
    • Paul E. McKenney's avatar
      rcutorture: Add trivial RCU implementation · c682db55
      Paul E. McKenney authored
      I have been showing off a trivial RCU implementation for non-preemptive
      environments for some time now:
      
      	#define rcu_read_lock()
      	#define rcu_read_unlock()
      	#define rcu_dereference(p) READ_ONCE(p)
      	#define rcu_assign_pointer(p, v) smp_store_release(&(p), (v))
      	void synchronize_rcu(void)
      	{
      	int cpu;
      		for_each_online_cpu(cpu)
      			sched_setaffinity(current->pid, cpumask_of(cpu));
      	}
      
      Trivial or not, as the old saying goes, "if it ain't tested, it don't
      work!".  This commit therefore adds a "trivial" flavor to rcutorture
      and a corresponding TRIVIAL test scenario.  This variant does not handle
      CPU hotplug, which is unconditionally enabled on x86 for post-v5.1-rc3
      kernels, which is why the TRIVIAL.boot says "rcutorture.onoff_interval=0".
      This commit actually does handle CONFIG_PREEMPT=y kernels, but only
      because it turns back the Linux-kernel clock in order to provide these
      alternative definitions (or the moral equivalent thereof):
      
      	#define rcu_read_lock() preempt_disable()
      	#define rcu_read_unlock() preempt_enable()
      
      In CONFIG_PREEMPT=n kernels without debugging, these are equivalent to
      empty macros give or take a compiler barrier.  However, the have been
      successfully tested with actual empty macros as well.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      [ paulmck: Fix symbol issue reported by kbuild test robot <lkp@intel.com>. ]
      [ paulmck: Work around sched_setaffinity() issue noted by Andrea Parri. ]
      [ paulmck: Add rcutorture.shuffle_interval=0 to TRIVIAL.boot to fix
        interaction with shuffler task noted by Peter Zijlstra. ]
      Tested-by: default avatarAndrea Parri <andrea.parri@amarulasolutions.com>
      c682db55
    • Paul E. McKenney's avatar
      rcutorture: Halt forward-progress checks at end of run · 3432d765
      Paul E. McKenney authored
      Once removed, an rcu_torture element can be deferred-freed by a chain
      of call_rcu() invocations, with each callback invoking another round of
      call_rcu() until either a fixed number of call_rcu() invocations have
      been chained or until the test ends.  This means that if the test ends,
      some of the rcu_torture elements will be "stranded" partway through the
      deferred-free process, which results in false-positive warnings from
      rcu_torture_writer() due to lack of forward progress should the test
      end just at the end of a stutter interval.
      
      This commit therefore suppresses rcu_torture_writer()'s forward-progress
      checks when the test ends in order to avoid these false-positive reports..
      Reported-by: default avatarSebastian Andrzej Siewior <bigeasy@linutronix.de>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      3432d765
    • Paul E. McKenney's avatar
      rcutorture: Give the scheduler a chance on PREEMPT && NO_HZ_FULL kernels · ab21f608
      Paul E. McKenney authored
      In !PREEMPT kernels, cond_resched() is a no-op.  In NO_HZ_FULL kernels,
      in-kernel execution (such as that of rcutorture's kthreads) might extend
      indefinitely without the scheduler gaining the aid of a scheduling-clock
      interrupt.  This combination can make the interaction of an rcutorture
      forward-progress test and a CPU-hotplug stop_machine operation make less
      forward progress than one might like.  Additionally, Sebastian Siewior
      notes that NO_HZ_FULL kernels have a scheduler check upon return to
      userspace execution, which suggests that in-kernel emulation of tight
      userspace loops containing system calls doing call_rcu() might also need
      explicit checks in the PREEMPT && NO_HZ_FULL case.
      
      This commit therefore introduces a rcu_torture_fwd_prog_cond_resched()
      function that explicitly invokes schedule() in such kernels whenever
      need_resched() returns true, while retaining use of cond_resched()
      for kernels that are either !PREEMPT or !NO_HZ_FULL.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      ab21f608
    • Paul E. McKenney's avatar
      rcutorture: Exempt TREE01 from forward-progress testing · 52b23be7
      Paul E. McKenney authored
      Because TREE01 can end up running more vCPUs that physical CPUs,
      hammering these shortchanged CPUs with tight loops containing call_rcu()
      invocations seems a bit like overkill.  This commit therefore exempts
      TREE01 from rcutorture's forward-progress testing.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      52b23be7
    • Paul E. McKenney's avatar
      rcutorture: Exempt tasks RCU from timely draining of grace periods · 5eabea59
      Paul E. McKenney authored
      After the end of each stutter pause interval, the rcu_torture_writer()
      kthread checks to be sure that all prior callbacks have completed so
      that all the test structures have been freed.  This works fine except
      for tasks RCU, in which grace periods can take one good long time.
      This commit therefore exempts tasks RCU from this check.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      5eabea59
    • Paul E. McKenney's avatar
      rcutorture: Provide rudimentary Makefile · 2456a856
      Paul E. McKenney authored
      This commit provides a rudimentary Makefile that runs a 10-minute
      rcutorture test on scenario TREE01.  This must be run on a system capable
      of spawning virtual machines and with everything installed to permit
      building Linux kernels.
      Reported-by: default avatarShuah Khan <shuah@kernel.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      2456a856
    • Paul E. McKenney's avatar
      torture: Make kvm-find-errors.sh and kvm-recheck.sh provide exit status · 63b29eae
      Paul E. McKenney authored
      This commit causes both kvm-find-errors.sh and kvm-recheck.sh to provide
      an exit status based on whether or not errors were located.  In the
      case of kvm-recheck.sh, this will be the error status of the last run.
      This change allows these commands to be used in scripting and Makefiles
      to automatically report failed rcutorture runs.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      63b29eae
    • Paul E. McKenney's avatar
      torture: Allow inter-stutter interval to be specified · ff3bf92d
      Paul E. McKenney authored
      Currently, the inter-stutter interval is the same as the stutter duration,
      that is, whatever number of jiffies is passed into torture_stutter_init().
      This has worked well for quite some time, but the addition of
      forward-progress testing to rcutorture can delay processes for several
      seconds, which can triple the time that they are stuttered.
      
      This commit therefore adds a second argument to torture_stutter_init()
      that specifies the inter-stutter interval.  While locktorture preserves
      the current behavior, rcutorture uses the RCU CPU stall warning interval
      to provide a wider inter-stutter interval.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      ff3bf92d
    • Paul E. McKenney's avatar
      rcutorture: Fix stutter_wait() return value and freelist checks · e8516c64
      Paul E. McKenney authored
      The stutter_wait() function is supposed to return true if it actually
      waits and false otherwise, but it instead unconditionally returns false.
      Which hides a bug in rcu_torture_writer() that fails to account for
      the fact that one of the rcu_tortures[] array elements will normally be
      referenced by rcu_torture_current, and thus not be on the freelist.
      
      This commit therefore corrects the stutter_wait() return value and adds a
      check for rcu_torture_current to rcu_torture_writer()'s check that things
      get freed after everything goes quiescent.  In addition, this commit
      causes torture_stutter() to give a bit more than one second (instead of
      only one jiffy) warning of the end of the stutter interval.  Finally,
      this commit disables long-delay readers and aggressive update-side
      forward-progress checks while forward-progress testing is in flight.
      Reported-by: default avatarSebastian Andrzej Siewior <bigeasy@linutronix.de>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      e8516c64
    • Paul E. McKenney's avatar
      rcutorture: Add cond_resched() to forward-progress free-up loop · 140e53f2
      Paul E. McKenney authored
      The rcu_torture_fwd_prog_cbfree() function frees callbacks used during
      rcutorture's call_rcu() forward-progress test, but does so in a tight
      loop.  This could cause problems given a very long list of callbacks to be
      freed, and actual testing produces lists with as many as 25M callbacks.
      This commit therefore adds a cond_resched() to this loop.  While in
      the area, this commit also rearranges the lock releases to look a bit
      more sane.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      140e53f2
    • Joel Fernandes (Google)'s avatar
      rcutorture: Add cpu0 to the set of CPUs to add jitter · dd064c35
      Joel Fernandes (Google) authored
      jitter.sh currently does not add CPU0 to the list of CPUs for adding of
      jitter. Let us add it to this list even when it is not hot-pluggable.
      Signed-off-by: default avatarJoel Fernandes (Google) <joel@joelfernandes.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      dd064c35
    • Joel Fernandes (Google)'s avatar
      rcutorture: Select from only online CPUs · e0e2147c
      Joel Fernandes (Google) authored
      The rcutorture jitter.sh script selects a random CPU but does not check
      if it is offline or online. This leads to taskset errors many times. On
      my machine, hyper threading is disabled so half the cores are offline
      causing taskset errors a lot of times. Let us fix this by checking from
      only the online CPUs on the system.
      
      Cc: rcu@vger.kernel.org
      Signed-off-by: default avatarJoel Fernandes (Google) <joel@joelfernandes.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      e0e2147c
    • Oleg Nesterov's avatar
      rcu/sync: Simplify the state machine · 89da3b94
      Oleg Nesterov authored
      With this patch rcu_sync has a single state variable and the transition rules
      become really simple:
      
      	GP_IDLE   - owned by the first rcu_sync_enter() which moves it to
      
      	GP_ENTER  - owned by rcu-callback which moves it to
      
      	GP_PASSED - owned by the last rcu_sync_exit() which moves it to
      
      	GP_EXIT   - and this is the only "nontrivial" state.
      
      		rcu-callback moves it back to GP_IDLE unless another enter()
      		comes before a GP pass.
      
      		If rcu-callback is invoked before the next rcu_sync_exit() it
      		must see gp_count incremented by that enter() and set GP_PASSED.
      
      		Otherwise, if the next rcu_sync_exit() wins the race, it will
      		move it to
      
      	GP_REPLAY - owned by rcu-callback which moves it to GP_EXIT
      Signed-off-by: default avatarOleg Nesterov <oleg@redhat.com>
      [ paulmck: While here, apply READ_ONCE() and WRITE_ONCE() to ->gp_state. ]
      [ paulmck: Tweaks to make htmldocs happy. (Reported by kbuild test robot.) ]
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      89da3b94
    • Oleg Nesterov's avatar
      locking/percpu-rwsem: Add DEFINE_PERCPU_RWSEM(), use it to initialize cgroup_threadgroup_rwsem · 3f2947b7
      Oleg Nesterov authored
      Turn DEFINE_STATIC_PERCPU_RWSEM() into __DEFINE_PERCPU_RWSEM() with the
      additional "is_static" argument to introduce DEFINE_PERCPU_RWSEM().
      
      Change cgroup.c to use DEFINE_PERCPU_RWSEM(cgroup_threadgroup_rwsem).
      Signed-off-by: default avatarOleg Nesterov <oleg@redhat.com>
      Reviewed-by: default avatarIngo Molnar <mingo@kernel.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      3f2947b7
    • Oleg Nesterov's avatar
      uprobes: Use DEFINE_STATIC_PERCPU_RWSEM() to initialize dup_mmap_sem · 2bf1acc2
      Oleg Nesterov authored
      Use DEFINE_STATIC_PERCPU_RWSEM() to initialize dup_mmap_sem.
      Signed-off-by: default avatarOleg Nesterov <oleg@redhat.com>
      Reviewed-by: default avatarIngo Molnar <mingo@kernel.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      2bf1acc2
    • Oleg Nesterov's avatar
      rcu/sync: Kill rcu_sync_type/gp_type · 95bf33b5
      Oleg Nesterov authored
      Now that the RCU flavors have been consolidated, rcu_sync_type makes no
      sense because none of internal update functions aside from .held() depend
      on gp_type.  This commit therefore removes this field and consolidates
      the relevant code.
      Signed-off-by: default avatarOleg Nesterov <oleg@redhat.com>
      [ paulmck: Added RCU and RCU-bh checks to rcu_sync_is_idle(). ]
      [ paulmck: And applied subsequent feedback from Oleg Nesterov. ]
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      95bf33b5
    • Jiang Biao's avatar
      rcu: Make __call_srcu static · 11b00045
      Jiang Biao authored
      Because __call_srcu() is not used outside kernel/rcu/srcutree.c,
      this commit makes it static.
      Signed-off-by: default avatarJiang Biao <benbjiang@tencent.com>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      11b00045
    • Joel Fernandes (Google)'s avatar
      module: Make srcu_struct ptr array as read-only · 056b89e7
      Joel Fernandes (Google) authored
      Since commit title ("srcu: Allocate per-CPU data for DEFINE_SRCU() in
      modules"), modules that call DEFINE_{STATIC,}SRCU will have a new array
      of srcu_struct pointers, which is used by srcu code to initialize and
      clean up these structures and save valuable per-cpu reserved space.
      
      There is no reason for this array of pointers to be writable, and can
      cause security or other hidden bugs. Mark these are read-only after the
      module init has completed.
      
      Tested with the following diff to ensure array not writable:
      
      (diff is a bit reduced to avoid patch command getting confused)
       a/kernel/module.c
       b/kernel/module.c
        -3506,6 +3506,14  static noinline int do_init_module [snip]
       	rcu_assign_pointer(mod->kallsyms, &mod->core_kallsyms);
       #endif
       	module_enable_ro(mod, true);
      +
      +	if (mod->srcu_struct_ptrs) {
      +		// Check if srcu_struct_ptrs access is possible
      +		char x = *(char *)mod->srcu_struct_ptrs;
      +		*(char *)mod->srcu_struct_ptrs = 0;
      +		*(char *)mod->srcu_struct_ptrs = x;
      +	}
      +
       	mod_tree_remove_init(mod);
       	disable_ro_nx(&mod->init_layout);
       	module_arch_freeing_init(mod);
      
      Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk>
      Cc: paulmck@linux.vnet.ibm.com
      Cc: rostedt@goodmis.org
      Cc: mathieu.desnoyers@efficios.com
      Cc: rcu@vger.kernel.org
      Cc: kernel-hardening@lists.openwall.com
      Cc: kernel-team@android.com
      Signed-off-by: default avatarJoel Fernandes (Google) <joel@joelfernandes.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      056b89e7
    • Joel Fernandes (Google)'s avatar
      srcu: Remove unused vmlinux srcu linker entries · 54e6c11b
      Joel Fernandes (Google) authored
      The SRCU for modules optimization (commit title "srcu: Allocate per-CPU
      data for DEFINE_SRCU() in modules") introduced vmlinux linker entries
      which is unused since it applies only to the built-in vmlinux. So remove
      it to prevent any space usage due to the 8 byte alignment it added.
      vmlinux.lds.h has no effect on module loading and is not used for
      building the module object, so the changes were not needed in the first
      place since the optimization is specific to modules.
      
      Tested with SRCU torture_type and rcutorture. Put prints in module
      loader to confirm it is able to find and initialize the srcu structures.
      
      Cc: Josh Triplett <josh@joshtriplett.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
      Cc: Lai Jiangshan <jiangshanlai@gmail.com>
      Cc: kernel-team@android.com
      Cc: paulmck@linux.vnet.ibm.com
      Signed-off-by: default avatarJoel Fernandes (Google) <joel@joelfernandes.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      54e6c11b
    • Paul E. McKenney's avatar
      srcu: Allocate per-CPU data for DEFINE_SRCU() in modules · fe15b50c
      Paul E. McKenney authored
      Adding DEFINE_SRCU() or DEFINE_STATIC_SRCU() to a loadable module requires
      that the size of the reserved region be increased, which is not something
      we want to be doing all that often.  One approach would be to require
      that loadable modules define an srcu_struct and invoke init_srcu_struct()
      from their module_init function and cleanup_srcu_struct() from their
      module_exit function.  However, this is more than a bit user unfriendly.
      
      This commit therefore creates an ___srcu_struct_ptrs linker section,
      and pointers to srcu_struct structures created by DEFINE_SRCU() and
      DEFINE_STATIC_SRCU() within a module are placed into that module's
      ___srcu_struct_ptrs section.  The required init_srcu_struct() and
      cleanup_srcu_struct() functions are then automatically invoked as needed
      when that module is loaded and unloaded, thus allowing modules to continue
      to use DEFINE_SRCU() and DEFINE_STATIC_SRCU() while avoiding the need
      to increase the size of the reserved region.
      
      Many of the algorithms and some of the code was cheerfully cherry-picked
      from other code making use of linker sections, perhaps most notably from
      tracepoints.  All bugs are nevertheless the sole property of the author.
      Suggested-by: default avatarMathieu Desnoyers <mathieu.desnoyers@efficios.com>
      [ paulmck: Use __section() and use "default" in srcu_module_notify()'s
        "switch" statement as suggested by Joel Fernandes. ]
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      Tested-by: default avatarJoel Fernandes (Google) <joel@joelfernandes.org>
      fe15b50c
    • Paul E. McKenney's avatar
    • Paul E. McKenney's avatar
      rcu: Set a maximum limit for back-to-back callback invocation · d5a9a8c3
      Paul E. McKenney authored
      Currently, if a CPU has more than 10,000 callbacks pending, it will
      increase rdp->blimit to LONG_MAX.  If you are lucky, LONG_MAX is only
      about two billion, but this is still a bit too many callbacks to invoke
      back-to-back while otherwise ignoring the world.
      
      This commit therefore sets a maximum limit of DEFAULT_MAX_RCU_BLIMIT,
      which is set to 10,000, for rdp->blimit.
      Reported-by: default avatarSebastian Andrzej Siewior <bigeasy@linutronix.de>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      d5a9a8c3
    • Paul E. McKenney's avatar
      rcu: Make kfree_rcu() ignore NULL pointers · 12edff04
      Paul E. McKenney authored
      This commit makes the kfree_rcu() macro's semantics be consistent
      with the likes of kfree() by adding a check for NULL pointers, so
      that kfree_rcu(NULL, ...) is a no-op.
      Reported-by: default avatarAndriy Shevchenko <andriy.shevchenko@linux.intel.com>
      Reported-by: default avatarChristoph Hellwig <hch@infradead.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      Reviewed-by: default avatarAndriy Shevchenko <andriy.shevchenko@linux.intel.com>
      12edff04
    • Neeraj Upadhyay's avatar
      rcu: Correctly unlock root node in rcu_check_gp_start_stall() · 3ae976a7
      Neeraj Upadhyay authored
      On systems whose rcu_node tree has only one node, the
      rcu_check_gp_start_stall() function's values of rnp and rnp_root will
      be identical.  In this case, it clearly does not make sense to release
      both rnp->lock and rnp_root->lock, but that is exactly what this function
      does in the last early exit.  This commit therefore unlocks only rnp->lock
      when rnp and rnp_root are equal.
      Signed-off-by: default avatarNeeraj Upadhyay <neeraju@codeaurora.org>
      Reviewed-by: default avatarMukesh Ojha <mojha@codeaurora.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      3ae976a7
    • Neeraj Upadhyay's avatar
      rcu: Dump specified number of blocked tasks · cd6d17b4
      Neeraj Upadhyay authored
      The dump_blkd_tasks() function dumps at most 10 blocked tasks, ignoring
      the value of the ncheck parameter.  This commit therefore substitutes
      the value of ncheck for the hard-coded value of 10.  Because all callers
      currently pass 10 as the number, this patch does not change behavior,
      but it is clearly an accident waiting to happen.
      Signed-off-by: default avatarNeeraj Upadhyay <neeraju@codeaurora.org>
      Reviewed-by: default avatarMukesh Ojha <mojha@codeaurora.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      cd6d17b4
    • Zhenzhong Duan's avatar
      doc: Fixup definition of rcupdate.rcu_task_stall_timeout · 588759a3
      Zhenzhong Duan authored
      A positive value of rcupdate.rcu_task_stall_timeout is an interval
      in seconds rather than jiffies.
      Signed-off-by: default avatarZhenzhong Duan <zhenzhong.duan@oracle.com>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
      588759a3