• Milton Miller's avatar
    call_function_many: add missing ordering · 45a57919
    Milton Miller authored
    Paul McKenney's review pointed out two problems with the barriers in the
    2.6.38 update to the smp call function many code.
    
    First, a barrier that would force the func and info members of data to
    be visible before their consumption in the interrupt handler was
    missing.  This can be solved by adding a smp_wmb between setting the
    func and info members and setting setting the cpumask; this will pair
    with the existing and required smp_rmb ordering the cpumask read before
    the read of refs.  This placement avoids the need a second smp_rmb in
    the interrupt handler which would be executed on each of the N cpus
    executing the call request.  (I was thinking this barrier was present
    but was not).
    
    Second, the previous write to refs (establishing the zero that we the
    interrupt handler was testing from all cpus) was performed by a third
    party cpu.  This would invoke transitivity which, as a recient or
    concurrent addition to memory-barriers.txt now explicitly states, would
    require a full smp_mb().
    
    However, we know the cpumask will only be set by one cpu (the data
    owner) and any preivous iteration of the mask would have cleared by the
    reading cpu.  By redundantly writing refs to 0 on the owning cpu before
    the smp_wmb, the write to refs will follow the same path as the writes
    that set the cpumask, which in turn allows us to keep the barrier in the
    interrupt handler a smp_rmb instead of promoting it to a smp_mb (which
    will be be executed by N cpus for each of the possible M elements on the
    list).
    
    I moved and expanded the comment about our (ab)use of the rcu list
    primitives for the concurrent walk earlier into this function.  I
    considered moving the first two paragraphs to the queue list head and
    lock, but felt it would have been too disconected from the code.
    
    Cc: Paul McKinney <paulmck@linux.vnet.ibm.com>
    Cc: stable@kernel.org (2.6.32 and later)
    Signed-off-by: default avatarMilton Miller <miltonm@bga.com>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    45a57919
smp.c 16.6 KB