• Nick Piggin's avatar
    x86: FIFO ticket spinlocks · 314cdbef
    Nick Piggin authored
    Introduce ticket lock spinlocks for x86 which are FIFO. The implementation
    is described in the comments. The straight-line lock/unlock instruction
    sequence is slightly slower than the dec based locks on modern x86 CPUs,
    however the difference is quite small on Core2 and Opteron when working out of
    cache, and becomes almost insignificant even on P4 when the lock misses cache.
    trylock is more significantly slower, but they are relatively rare.
    
    On an 8 core (2 socket) Opteron, spinlock unfairness is extremely noticable,
    with a userspace test having a difference of up to 2x runtime per thread, and
    some threads are starved or "unfairly" granted the lock up to 1 000 000 (!)
    times. After this patch, all threads appear to finish at exactly the same
    time.
    
    The memory ordering of the lock does conform to x86 standards, and the
    implementation has been reviewed by Intel and AMD engineers.
    
    The algorithm also tells us how many CPUs are contending the lock, so
    lockbreak becomes trivial and we no longer have to waste 4 bytes per
    spinlock for it.
    
    After this, we can no longer spin on any locks with preempt enabled
    and cannot reenable interrupts when spinning on an irq safe lock, because
    at that point we have already taken a ticket and the would deadlock if
    the same CPU tries to take the lock again.  These are questionable anyway:
    if the lock happens to be called under a preempt or interrupt disabled section,
    then it will just have the same latency problems. The real fix is to keep
    critical sections short, and ensure locks are reasonably fair (which this
    patch does).
    Signed-off-by: default avatarNick Piggin <npiggin@suse.de>
    Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
    Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
    314cdbef
paravirt.h 30.5 KB