• Jason Gunthorpe's avatar
    mm/hmm: Fix error flows in hmm_invalidate_range_start · 5a136b4a
    Jason Gunthorpe authored
    
    
    If the trylock on the hmm->mirrors_sem fails the function will return
    without decrementing the notifiers that were previously incremented. Since
    the caller will not call invalidate_range_end() on EAGAIN this will result
    in notifiers becoming permanently incremented and deadlock.
    
    If the sync_cpu_device_pagetables() required blocking the function will
    not return EAGAIN even though the device continues to touch the
    pages. This is a violation of the mmu notifier contract.
    
    Switch, and rename, the ranges_lock to a spin lock so we can reliably
    obtain it without blocking during error unwind.
    
    The error unwind is necessary since the notifiers count must be held
    incremented across the call to sync_cpu_device_pagetables() as we cannot
    allow the range to become marked valid by a parallel
    invalidate_start/end() pair while doing sync_cpu_device_pagetables().
    Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
    Reviewed-by: default avatarRalph Campbell <rcampbell@nvidia.com>
    Reviewed-by: default avatarChristoph Hellwig <hch@lst.de>
    Tested-by: default avatarPhilip Yang <Philip.Yang@amd.com>
    5a136b4a
hmm.c 42.1 KB