• David Matlack's avatar
    KVM: x86/mmu: Always drop mmu_lock to allocate TDP MMU SPs for eager splitting · cf3ff0ee
    David Matlack authored
    Always drop mmu_lock to allocate shadow pages in the TDP MMU when doing
    eager page splitting. Dropping mmu_lock during eager page splitting is
    cheap since KVM does not have to flush remote TLBs, and avoids stalling
    vCPU threads that are taking page faults while KVM is eager splitting
    under mmu_lock held for write.
    
    This change reduces 20%+ dips in MySQL throughput during live migration
    in a 160 vCPU VM while userspace is issuing CLEAR_DIRTY_LOG ioctls
    (tested with 1GiB and 8GiB CLEARs). Userspace could issue finer-grained
    CLEARs, which would also reduce contention on mmu_lock, but doing so
    will increase the rate of remote TLB flushing, since KVM must flush TLBs
    before returning from CLEAR_DITY_LOG.
    
    When there isn't contention on mmu_lock[1], this change does not regress
    the time it takes to perform eager page splitting (the cost of releasing
    and re-acquiring an uncontended lock is minimal on x86).
    
    [1] Tested with dirty_log_perf_test, which does not run vCPUs during
    eager page splitting, and with a 16 vCPU VM Live Migration with
    manual-protect disabled (where mmu_lock is held for read).
    
    Cc: Bibo Mao <maobibo@loongson.cn>
    Cc: Sean Christopherson <seanjc@google.com>
    Signed-off-by: default avatarDavid Matlack <dmatlack@google.com>
    Link: https://lore.kernel.org/r/20240611220512.2426439-2-dmatlack@google.comSigned-off-by: default avatarSean Christopherson <seanjc@google.com>
    cf3ff0ee
tdp_mmu.c 54.3 KB