• Frederic Weisbecker's avatar
    rcu: Remove full ordering on second EQS snapshot · 0a5e9bd3
    Frederic Weisbecker authored
    When the grace period kthread checks the extended quiescent state
    counter of a CPU, full ordering is necessary to ensure that either:
    
    * If the GP kthread observes the remote target in an extended quiescent
      state, then that target must observe all accesses prior to the current
      grace period, including the current grace period sequence number, once
      it exits that extended quiescent state. Also the GP kthread must
      observe all accesses performed by the target prior it entering in
      EQS.
    
    or:
    
    * If the GP kthread observes the remote target NOT in an extended
      quiescent state, then the target further entering in an extended
      quiescent state must observe all accesses prior to the current
      grace period, including the current grace period sequence number, once
      it enters that extended quiescent state. Also the GP kthread later
      observing that EQS must also observe all accesses performed by the
      target prior it entering in EQS.
    
    This ordering is explicitly performed both on the first EQS snapshot
    and on the second one as well through the combination of a preceding
    full barrier followed by an acquire read. However the second snapshot's
    full memory barrier is redundant and not needed to enforce the above
    guarantees:
    
        GP kthread                  Remote target
        ----                        -----
        // Access prior GP
        WRITE_ONCE(A, 1)
        // first snapshot
        smp_mb()
        x = smp_load_acquire(EQS)
                                   // Access prior GP
                                   WRITE_ONCE(B, 1)
                                   // EQS enter
                                   // implied full barrier by atomic_add_return()
                                   atomic_add_return(RCU_DYNTICKS_IDX, EQS)
                                   // implied full barrier by atomic_add_return()
                                   READ_ONCE(A)
        // second snapshot
        y = smp_load_acquire(EQS)
        z = READ_ONCE(B)
    
    If the GP kthread above fails to observe the remote target in EQS
    (x not in EQS), the remote target will observe A == 1 after further
    entering in EQS. Then the second snapshot taken by the GP kthread only
    need to be an acquire read in order to observe z == 1.
    
    Therefore remove the needless full memory barrier on second snapshot.
    Signed-off-by: default avatarFrederic Weisbecker <frederic@kernel.org>
    Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
    Reviewed-by: default avatarBoqun Feng <boqun.feng@gmail.com>
    Reviewed-by: default avatarNeeraj Upadhyay <neeraj.upadhyay@kernel.org>
    Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
    0a5e9bd3
tree.c 183 KB