• Davidlohr Bueso's avatar
    ipc/sem: rework task wakeups · 9ae949fa
    Davidlohr Bueso authored
    Our sysv sems have been using the notion of lockless wakeups for a
    while, ever since commit 0a2b9d4c ("ipc/sem.c: move wake_up_process
    out of the spinlock section"), in order to reduce the sem_lock hold
    times.  This in-house pending queue can be replaced by wake_q (just like
    all the rest of ipc now), in that it provides the following advantages:
    
     o Simplifies and gets rid of unnecessary code.
    
     o We get rid of the IN_WAKEUP complexities. Given that wake_q_add()
       grabs reference to the task, if awoken due to an unrelated event,
       between the wake_q_add() and wake_up_q() window, we cannot race with
       sys_exit and the imminent call to wake_up_process().
    
     o By not spinning IN_WAKEUP, we no longer need to disable preemption.
    
    In consequence, the wakeup paths (after schedule(), that is) must
    acknowledge an external signal/event, as well spurious wakeup occurring
    during the pending wakeup window.  Obviously no changes in semantics
    that could be visible to the user.  The fastpath is _only_ for when we
    know for sure that we were awoken due to a the waker's successful semop
    call (queue.status is not -EINTR).
    
    On a 48-core Haswell, running the ipcscale 'waitforzero' test, the
    following is seen with increasing thread counts:
    
                                   v4.8-rc5                v4.8-rc5
                                                            semopv2
    Hmean    sembench-sem-2      574733.00 (  0.00%)   578322.00 (  0.62%)
    Hmean    sembench-sem-8      811708.00 (  0.00%)   824689.00 (  1.59%)
    Hmean    sembench-sem-12     842448.00 (  0.00%)   845409.00 (  0.35%)
    Hmean    sembench-sem-21     933003.00 (  0.00%)   977748.00 (  4.80%)
    Hmean    sembench-sem-48     935910.00 (  0.00%)  1004759.00 (  7.36%)
    Hmean    sembench-sem-79     937186.00 (  0.00%)   983976.00 (  4.99%)
    Hmean    sembench-sem-234    974256.00 (  0.00%)  1060294.00 (  8.83%)
    Hmean    sembench-sem-265    975468.00 (  0.00%)  1016243.00 (  4.18%)
    Hmean    sembench-sem-296    991280.00 (  0.00%)  1042659.00 (  5.18%)
    Hmean    sembench-sem-327    975415.00 (  0.00%)  1029977.00 (  5.59%)
    Hmean    sembench-sem-358   1014286.00 (  0.00%)  1049624.00 (  3.48%)
    Hmean    sembench-sem-389    972939.00 (  0.00%)  1043127.00 (  7.21%)
    Hmean    sembench-sem-420    981909.00 (  0.00%)  1056747.00 (  7.62%)
    Hmean    sembench-sem-451    990139.00 (  0.00%)  1051609.00 (  6.21%)
    Hmean    sembench-sem-482    965735.00 (  0.00%)  1040313.00 (  7.72%)
    
    [akpm@linux-foundation.org: coding-style fixes]
    [sfr@canb.auug.org.au: merge fix for WAKE_Q to DEFINE_WAKE_Q rename]
      Link: http://lkml.kernel.org/r/20161122210410.5eca9fc2@canb.auug.org.au
    Link: http://lkml.kernel.org/r/1474225896-10066-3-git-send-email-dave@stgolabs.netSigned-off-by: default avatarDavidlohr Bueso <dbueso@suse.de>
    Acked-by: default avatarManfred Spraul <manfred@colorfullife.com>
    Signed-off-by: default avatarStephen Rothwell <sfr@canb.auug.org.au>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    9ae949fa
sem.c 53.4 KB