• Brice Goglin's avatar
    dmaengine: make dma_channel_rebalance() NUMA aware · c4d27c4d
    Brice Goglin authored
    dma_channel_rebalance() currently distributes channels by processor ID.
    These IDs often change with the BIOS, and the order isn't related to
    the DMA channel list (related to PCI bus ids).
    * On my SuperMicro dual E5 machine, first socket has processor IDs [0-7]
      (and [16-23] for hyperthreads), second socket has [8-15]+[24-31]
      => channels are properly allocated to local CPUs.
    * On Dells R720 with same processors, first socket has even processor IDs,
      second socket has odd numbers
      => half the processors get channels on the remote socket, causing
         cross-NUMA traffic and lower DMA performance.
    
    Change nth_chan() to return the channel with min table_count and in the
    NUMA node of the given CPU, if any. If none, the (non-local) channel with
    min table_count is returned. nth_chan() is therefore renamed into min_chan()
    since we don't iterate until the nth channel anymore. In practice, the
    behavior is the same because first channels are taken first and are then
    ignored because they got an additional reference.
    
    The new code has a slightly higher complexity since we always scan the
    entire list of channels for finding the minimal table_count (instead
    of stopping after N chans), and because we check whether the CPU is in the
    DMA device locality mask. Overall we still have time complexity =
    number of chans x number of processors. This rebalance is rarely used,
    so this won't hurt.
    
    On the above SuperMicro machine, channels are still allocated the same.
    On the Dells, there are no locality issue anymore (MEMCPY channel X goes
    to processor X and to its hyperthread sibling).
    Signed-off-by: default avatarBrice Goglin <Brice.Goglin@inria.fr>
    Signed-off-by: default avatarDan Williams <djbw@fb.com>
    c4d27c4d
dmaengine.c 29.3 KB