• Waiman Long's avatar
    locking/qrwlock: Reduce reader/writer to reader lock transfer latency · ffffeaf3
    Waiman Long authored
    Currently, a reader will check first to make sure that the writer mode
    byte is cleared before incrementing the reader count. That waiting is
    not really necessary. It increases the latency in the reader/writer
    to reader transition and reduces readers performance.
    
    This patch eliminates that waiting. It also has the side effect
    of reducing the chance of writer lock stealing and improving the
    fairness of the lock. Using a locking microbenchmark, a 10-threads 5M
    locking loop of mostly readers (RW ratio = 10,000:1) has the following
    performance numbers in a Haswell-EX box:
    
            Kernel          Locking Rate (Kops/s)
            ------          ---------------------
            4.1.1               15,063,081
            4.1.1+patch         17,241,552  (+14.4%)
    Signed-off-by: default avatarWaiman Long <Waiman.Long@hp.com>
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Cc: Arnd Bergmann <arnd@arndb.de>
    Cc: Douglas Hatch <doug.hatch@hp.com>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Scott J Norton <scott.norton@hp.com>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Will Deacon <will.deacon@arm.com>
    Link: http://lkml.kernel.org/r/1436459543-29126-2-git-send-email-Waiman.Long@hp.comSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
    ffffeaf3
qrwlock.c 4.05 KB