• Shivaprasad G Bhat's avatar
    powerpc/iommu: Reimplement the iommu_table_group_ops for pSeries · f431a8cd
    Shivaprasad G Bhat authored
    PPC64 IOMMU API defines iommu_table_group_ops which handles DMA
    windows for PEs, their ownership transfer, create/set/unset the TCE
    tables for the Dynamic DMA wundows(DDW). VFIOS uses these APIs for
    support on POWER.
    
    The commit 9d67c943 ("powerpc/iommu: Add "borrowing"
    iommu_table_group_ops") implemented partial support for this API with
    "borrow" mechanism wherein the DMA windows if created already by the
    host driver, they would be available for VFIO to use. Also, it didn't
    have the support to control/modify the window size or the IO page
    size.
    
    The current patch implements all the necessary iommu_table_group_ops
    APIs there by avoiding the "borrrowing". So, just the way it is on the
    PowerNV platform, with this patch the iommu table group ownership is
    transferred to the VFIO PPC subdriver, the iommu table, DMA windows
    creation/deletion all driven through the APIs.
    
    The pSeries uses the query-pe-dma-window, create-pe-dma-window and
    reset-pe-dma-window RTAS calls for DMA window creation, deletion and
    reset to defaul. The RTAs calls do show some minor differences to the
    way things are to be handled on the pSeries which are listed below.
    
    * On pSeries, the default DMA window size is "fixed" cannot be custom
    sized as requested by the user. For non-SRIOV VFs, It is fixed at 2GB
    and for SRIOV VFs, its variable sized based on the capacity assigned
    to it during the VF assignment to the LPAR. So, for the  default DMA
    window alone the size if requested less than tce32_size, the smaller
    size is enforced using the iommu table->it_size.
    
    * The DMA start address for 32-bit window is 0, and for the 64-bit
    window in case of PowerNV is hardcoded to TVE select (bit 59) at 512PiB
    offset. This address is returned at the time of create_table() API call
    (even before the window is created), the subsequent set_window() call
    actually opens the DMA window. On pSeries, the DMA start address for
    32-bit window is known from the 'ibm,dma-window' DT property. However,
    the 64-bit window start address is not known until the create-pe-dma
    RTAS call is made. So, the create_table() which returns the DMA window
    start address actually opens the DMA window and returns the DMA start
    address as returned by the Hypervisor for the create-pe-dma RTAS call.
    
    * The reset-pe-dma RTAS call resets the DMA windows and restores the
    default DMA window, however it does not clear the TCE table entries
    if there are any. In case of ownership transfer from platform domain
    which used direct mapping, the patch chooses remove-pe-dma instead of
    reset-pe for the 64-bit window intentionally so that the
    clear_dma_window() is called.
    
    Other than the DMA window management changes mentioned above, the
    patch also brings back the userspace view for the single level TCE
    as it existed before commit 090bad39 ("powerpc/powernv: Add
    indirect levels to it_userspace") along with the relavent
    refactoring.
    Signed-off-by: default avatarShivaprasad G Bhat <sbhat@linux.ibm.com>
    Signed-off-by: default avatarMichael Ellerman <mpe@ellerman.id.au>
    Link: https://msgid.link/171923275958.1397.907964437142542242.stgit@linux.ibm.com
    f431a8cd
pci-ioda.c 75.1 KB