• Sean Christopherson's avatar
    KVM: x86/mmu: Reintroduce fast invalidate/zap for flushing memslot · 002c5f73
    Sean Christopherson authored
    James Harvey reported a livelock that was introduced by commit
    d012a06a ("Revert "KVM: x86/mmu: Zap only the relevant pages when
    removing a memslot"").
    
    The livelock occurs because kvm_mmu_zap_all() as it exists today will
    voluntarily reschedule and drop KVM's mmu_lock, which allows other vCPUs
    to add shadow pages.  With enough vCPUs, kvm_mmu_zap_all() can get stuck
    in an infinite loop as it can never zap all pages before observing lock
    contention or the need to reschedule.  The equivalent of kvm_mmu_zap_all()
    that was in use at the time of the reverted commit (4e103134, "KVM:
    x86/mmu: Zap only the relevant pages when removing a memslot") employed
    a fast invalidate mechanism and was not susceptible to the above livelock.
    
    There are three ways to fix the livelock:
    
    - Reverting the revert (commit d012a06a) is not a viable option as
      the revert is needed to fix a regression that occurs when the guest has
      one or more assigned devices.  It's unlikely we'll root cause the device
      assignment regression soon enough to fix the regression timely.
    
    - Remove the conditional reschedule from kvm_mmu_zap_all().  However, although
      removing the reschedule would be a smaller code change, it's less safe
      in the sense that the resulting kvm_mmu_zap_all() hasn't been used in
      the wild for flushing memslots since the fast invalidate mechanism was
      introduced by commit 6ca18b69 ("KVM: x86: use the fast way to
      invalidate all pages"), back in 2013.
    
    - Reintroduce the fast invalidate mechanism and use it when zapping shadow
      pages in response to a memslot being deleted/moved, which is what this
      patch does.
    
    For all intents and purposes, this is a revert of commit ea145aac
    ("Revert "KVM: MMU: fast invalidate all pages"") and a partial revert of
    commit 7390de1e ("Revert "KVM: x86: use the fast way to invalidate
    all pages""), i.e. restores the behavior of commit 5304b8d3 ("KVM:
    MMU: fast invalidate all pages") and commit 6ca18b69 ("KVM: x86:
    use the fast way to invalidate all pages") respectively.
    
    Fixes: d012a06a ("Revert "KVM: x86/mmu: Zap only the relevant pages when removing a memslot"")
    Reported-by: default avatarJames Harvey <jamespharvey20@gmail.com>
    Cc: Alex Willamson <alex.williamson@redhat.com>
    Cc: Paolo Bonzini <pbonzini@redhat.com>
    Cc: stable@vger.kernel.org
    Signed-off-by: default avatarSean Christopherson <sean.j.christopherson@intel.com>
    Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
    002c5f73
mmu.c 162 KB