• James Smart's avatar
    scsi: lpfc: Change default IRQ model on AMD architectures · dcaa2136
    James Smart authored
    The current driver attempts to allocate an interrupt vector per cpu using
    the systems managed IRQ allocator (flag PCI_IRQ_AFFINITY). The system IRQ
    allocator will either provide the per-cpu vector, or return fewer
    vectors. When fewer vectors, they are evenly spread between the numa nodes
    on the system.  When run on an AMD architecture, if interrupts occur to a
    cpu that is not in the same numa node as the adapter generating the
    interrupt, there are extreme costs and overheads in performance.  Thus, if
    1:1 vector allocation is used, or the "balanced" vectors in the other numa
    nodes, performance can be hit significantly.
    
    A much more performant model is to allocate interrupts only on the cpus
    that are in the numa node where the adapter resides.  I/O completion is
    still performed by the cpu where the I/O was generated. Unfortunately,
    there is no flag to request the managed IRQ subsystem allocate vectors only
    for the CPUs in the numa node as the adapter.
    
    On AMD architecture, revert the irq allocation to the normal style
    (non-managed) and then use irq_set_affinity_hint() to set the cpu
    affinity and disable user-space rebalancing.
    
    Tie the support into CPU offline/online. If the cpu being offlined owns a
    vector, the vector is re-affinitized to one of the other CPUs on the same
    numa node. If there are no more CPUs on the numa node, the vector has all
    affinity removed and lets the system determine where it's serviced.
    Similarly, when the cpu that owned a vector comes online, the vector is
    reaffinitized to the cpu.
    
    Link: https://lore.kernel.org/r/20191105005708.7399-10-jsmart2021@gmail.comSigned-off-by: default avatarDick Kennedy <dick.kennedy@broadcom.com>
    Signed-off-by: default avatarJames Smart <jsmart2021@gmail.com>
    Signed-off-by: default avatarMartin K. Petersen <martin.petersen@oracle.com>
    dcaa2136
lpfc_attr.c 220 KB