Commit 207bbfb6 authored by Shane Xiao's avatar Shane Xiao Committed by Alex Deucher

drm/amdgpu: Add userptr bo support for mGPUs when iommu is on

For userptr bo with iommu on, multiple GPUs use same system
memory dma mapping address when both adev and bo_adev are in
identity mode or in the same iommu group.

If RAM direct map to one GPU, other GPUs can share the original
BO in order to reduce dma address array usage when RAM can
direct map to these GPUs. However, we should explicit check
whether RAM can direct map to all these GPUs.

This patch fixes a potential issue that where RAM is
direct mapped on some but not all GPUs.

v2:
  1. Update comment
  2. Add helper function reuse_dmamap
Signed-off-by: default avatarShane Xiao <shane.xiao@amd.com>
Reviewed-by: default avatarFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: default avatarAlex Deucher <alexander.deucher@amd.com>
parent 11f25c84
...@@ -82,6 +82,25 @@ static bool kfd_mem_is_attached(struct amdgpu_vm *avm, ...@@ -82,6 +82,25 @@ static bool kfd_mem_is_attached(struct amdgpu_vm *avm,
return false; return false;
} }
/**
* reuse_dmamap() - Check whether adev can share the original
* userptr BO
*
* If both adev and bo_adev are in direct mapping or
* in the same iommu group, they can share the original BO.
*
* @adev: Device to which can or cannot share the original BO
* @bo_adev: Device to which allocated BO belongs to
*
* Return: returns true if adev can share original userptr BO,
* false otherwise.
*/
static bool reuse_dmamap(struct amdgpu_device *adev, struct amdgpu_device *bo_adev)
{
return (adev->ram_is_direct_mapped && bo_adev->ram_is_direct_mapped) ||
(adev->dev->iommu_group == bo_adev->dev->iommu_group);
}
/* Set memory usage limits. Current, limits are /* Set memory usage limits. Current, limits are
* System (TTM + userptr) memory - 15/16th System RAM * System (TTM + userptr) memory - 15/16th System RAM
* TTM memory - 3/8th System RAM * TTM memory - 3/8th System RAM
...@@ -805,11 +824,11 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, ...@@ -805,11 +824,11 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem,
va + bo_size, vm); va + bo_size, vm);
if ((adev == bo_adev && !(mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_MMIO_REMAP)) || if ((adev == bo_adev && !(mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_MMIO_REMAP)) ||
(amdgpu_ttm_tt_get_usermm(mem->bo->tbo.ttm) && adev->ram_is_direct_mapped) || (amdgpu_ttm_tt_get_usermm(mem->bo->tbo.ttm) && reuse_dmamap(adev, bo_adev)) ||
same_hive) { same_hive) {
/* Mappings on the local GPU, or VRAM mappings in the /* Mappings on the local GPU, or VRAM mappings in the
* local hive, or userptr mapping IOMMU direct map mode * local hive, or userptr mapping can reuse dma map
* share the original BO * address space share the original BO
*/ */
attachment[i]->type = KFD_MEM_ATT_SHARED; attachment[i]->type = KFD_MEM_ATT_SHARED;
bo[i] = mem->bo; bo[i] = mem->bo;
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment