• Alexey Kardashevskiy's avatar
    powerpc/powernv/npu: Simplify DMA setup · f9f83456
    Alexey Kardashevskiy authored
    NPU devices are emulated in firmware and mainly used for NPU NVLink
    training; one NPU device is per a hardware link. Their DMA/TCE setup
    must match the GPU which is connected via PCIe and NVLink so any changes
    to the DMA/TCE setup on the GPU PCIe device need to be propagated to
    the NVLink device as this is what device drivers expect and it doesn't
    make much sense to do anything else.
    
    This makes NPU DMA setup explicit.
    pnv_npu_ioda_controller_ops::pnv_npu_dma_set_mask is moved to pci-ioda,
    made static and prints warning as dma_set_mask() should never be called
    on this function as in any case it will not configure GPU; so we make
    this explicit.
    
    Instead of using PNV_IODA_PE_PEER and peers[] (which the next patch will
    remove), we test every PCI device if there are corresponding NVLink
    devices. If there are any, we propagate bypass mode to just found NPU
    devices by calling the setup helper directly (which takes @bypass) and
    avoid guessing (i.e. calculating from DMA mask) whether we need bypass
    or not on NPU devices. Since DMA setup happens in very rare occasion,
    this will not slow down booting or VFIO start/stop much.
    
    This renames pnv_npu_disable_bypass to pnv_npu_dma_set_32 to make it
    more clear what the function really does which is programming 32bit
    table address to the TVT ("disabling bypass" means writing zeroes to
    the TVT).
    
    This removes pnv_npu_dma_set_bypass() from pnv_npu_ioda_fixup() as
    the DMA configuration on NPU does not matter until dma_set_mask() is
    called on GPU and that will do the NPU DMA configuration.
    
    This removes phb->dma_dev_setup initialization for NPU as
    pnv_pci_ioda_dma_dev_setup is no-op for it anyway.
    
    This stops using npe->tce_bypass_base as it never changes and values
    other than zero are not supported.
    Signed-off-by: default avatarAlexey Kardashevskiy <aik@ozlabs.ru>
    Reviewed-by: default avatarDavid Gibson <david@gibson.dropbear.id.au>
    Reviewed-by: default avatarAlistair Popple <alistair@popple.id.au>
    Signed-off-by: default avatarMichael Ellerman <mpe@ellerman.id.au>
    f9f83456
pci-ioda.c 95.4 KB