• Doug Ledford's avatar
    IB/ipoib: Use dedicated workqueues per interface · 0b39578b
    Doug Ledford authored
    During my recent work on the rtnl lock deadlock in the IPoIB driver, I
    saw that even once I fixed the apparent races for a single device, as
    soon as that device had any children, new races popped up.  It turns
    out that this is because no matter how well we protect against races
    on a single device, the fact that all devices use the same workqueue,
    and flush_workqueue() flushes *everything* from that workqueue means
    that we would also have to prevent all races between different devices
    (for instance, ipoib_mcast_restart_task on interface ib0 can race with
    ipoib_mcast_flush_dev on interface ib0.8002, resulting in a deadlock on
    the rtnl_lock).
    
    There are several possible solutions to this problem:
    
    Make carrier_on_task and mcast_restart_task try to take the rtnl for
    some set period of time and if they fail, then bail.  This runs the
    real risk of dropping work on the floor, which can end up being its
    own separate kind of deadlock.
    
    Set some global flag in the driver that says some device is in the
    middle of going down, letting all tasks know to bail.  Again, this can
    drop work on the floor.
    
    Or the method this patch attempts to use, which is when we bring an
    interface up, create a workqueue specifically for that interface, so
    that when we take it back down, we are flushing only those tasks
    associated with our interface.  In addition, keep the global
    workqueue, but now limit it to only flush tasks.  In this way, the
    flush tasks can always flush the device specific work queues without
    having deadlock issues.
    Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
    0b39578b
ipoib_cm.c 42.3 KB