• Davidlohr Bueso's avatar
    locking/rwsem: Set lock ownership ASAP · 7a215f89
    Davidlohr Bueso authored
    In order to optimize the spinning step, we need to set the lock
    owner as soon as the lock is acquired; after a successful counter
    cmpxchg operation, that is. This is particularly useful as rwsems
    need to set the owner to nil for readers, so there is a greater
    chance of falling out of the spinning. Currently we only set the
    owner much later in the game, in the more generic level -- latency
    can be specially bad when waiting for a node->next pointer when
    releasing the osq in up_write calls.
    
    As such, update the owner inside rwsem_try_write_lock (when the
    lock is obtained after blocking) and rwsem_try_write_lock_unqueued
    (when the lock is obtained while spinning). This requires creating
    a new internal rwsem.h header to share the owner related calls.
    
    Also cleanup some headers for mutex and rwsem.
    Suggested-by: default avatarPeter Zijlstra <peterz@infradead.org>
    Signed-off-by: default avatarDavidlohr Bueso <dbueso@suse.de>
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Cc: Jason Low <jason.low2@hp.com>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Michel Lespinasse <walken@google.com>
    Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
    Cc: Tim Chen <tim.c.chen@linux.intel.com>
    Link: http://lkml.kernel.org/r/1422609267-15102-4-git-send-email-dave@stgolabs.netSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
    7a215f89
rwsem.h 357 Bytes