1. 08 Dec, 2021 16 commits
  2. 05 Dec, 2021 3 commits
  3. 02 Dec, 2021 4 commits
    • Sean Christopherson's avatar
      KVM: x86/mmu: Retry page fault if root is invalidated by memslot update · a955cad8
      Sean Christopherson authored
      Bail from the page fault handler if the root shadow page was obsoleted by
      a memslot update.  Do the check _after_ acuiring mmu_lock, as the TDP MMU
      doesn't rely on the memslot/MMU generation, and instead relies on the
      root being explicit marked invalid by kvm_mmu_zap_all_fast(), which takes
      mmu_lock for write.
      
      For the TDP MMU, inserting a SPTE into an obsolete root can leak a SP if
      kvm_tdp_mmu_zap_invalidated_roots() has already zapped the SP, i.e. has
      moved past the gfn associated with the SP.
      
      For other MMUs, the resulting behavior is far more convoluted, though
      unlikely to be truly problematic.  Installing SPs/SPTEs into the obsolete
      root isn't directly problematic, as the obsolete root will be unloaded
      and dropped before the vCPU re-enters the guest.  But because the legacy
      MMU tracks shadow pages by their role, any SP created by the fault can
      can be reused in the new post-reload root.  Again, that _shouldn't_ be
      problematic as any leaf child SPTEs will be created for the current/valid
      memslot generation, and kvm_mmu_get_page() will not reuse child SPs from
      the old generation as they will be flagged as obsolete.  But, given that
      continuing with the fault is pointess (the root will be unloaded), apply
      the check to all MMUs.
      
      Fixes: b7cccd39 ("KVM: x86/mmu: Fast invalidation for TDP MMU")
      Cc: stable@vger.kernel.org
      Cc: Ben Gardon <bgardon@google.com>
      Signed-off-by: default avatarSean Christopherson <seanjc@google.com>
      Message-Id: <20211120045046.3940942-5-seanjc@google.com>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      a955cad8
    • Dan Carpenter's avatar
      KVM: VMX: Set failure code in prepare_vmcs02() · bfbb307c
      Dan Carpenter authored
      The error paths in the prepare_vmcs02() function are supposed to set
      *entry_failure_code but this path does not.  It leads to using an
      uninitialized variable in the caller.
      
      Fixes: 71f73470 ("KVM: nVMX: Load GUEST_IA32_PERF_GLOBAL_CTRL MSR on VM-Entry")
      Signed-off-by: default avatarDan Carpenter <dan.carpenter@oracle.com>
      Message-Id: <20211130125337.GB24578@kili>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      bfbb307c
    • Paolo Bonzini's avatar
      KVM: ensure APICv is considered inactive if there is no APIC · ef8b4b72
      Paolo Bonzini authored
      kvm_vcpu_apicv_active() returns false if a virtual machine has no in-kernel
      local APIC, however kvm_apicv_activated might still be true if there are
      no reasons to disable APICv; in fact it is quite likely that there is none
      because APICv is inhibited by specific configurations of the local APIC
      and those configurations cannot be programmed.  This triggers a WARN:
      
         WARN_ON_ONCE(kvm_apicv_activated(vcpu->kvm) != kvm_vcpu_apicv_active(vcpu));
      
      To avoid this, introduce another cause for APICv inhibition, namely the
      absence of an in-kernel local APIC.  This cause is enabled by default,
      and is dropped by either KVM_CREATE_IRQCHIP or the enabling of
      KVM_CAP_IRQCHIP_SPLIT.
      Reported-by: default avatarIgnat Korchagin <ignat@cloudflare.com>
      Fixes: ee49a893 ("KVM: x86: Move SVM's APICv sanity check to common x86", 2021-10-22)
      Reviewed-by: default avatarMaxim Levitsky <mlevitsk@redhat.com>
      Reviewed-by: default avatarSean Christopherson <seanjc@google.com>
      Tested-by: default avatarIgnat Korchagin <ignat@cloudflare.com>
      Message-Id: <20211130123746.293379-1-pbonzini@redhat.com>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      ef8b4b72
    • Like Xu's avatar
      KVM: x86/pmu: Fix reserved bits for AMD PerfEvtSeln register · cb1d220d
      Like Xu authored
      If we run the following perf command in an AMD Milan guest:
      
        perf stat \
        -e cpu/event=0x1d0/ \
        -e cpu/event=0x1c7/ \
        -e cpu/umask=0x1f,event=0x18e/ \
        -e cpu/umask=0x7,event=0x18e/ \
        -e cpu/umask=0x18,event=0x18e/ \
        ./workload
      
      dmesg will report a #GP warning from an unchecked MSR access
      error on MSR_F15H_PERF_CTLx.
      
      This is because according to APM (Revision: 4.03) Figure 13-7,
      the bits [35:32] of AMD PerfEvtSeln register is a part of the
      event select encoding, which extends the EVENT_SELECT field
      from 8 bits to 12 bits.
      
      Opportunistically update pmu->reserved_bits for reserved bit 19.
      Reported-by: default avatarJim Mattson <jmattson@google.com>
      Fixes: ca724305 ("KVM: x86/vPMU: Implement AMD vPMU code for KVM")
      Signed-off-by: default avatarLike Xu <likexu@tencent.com>
      Message-Id: <20211118130320.95997-1-likexu@tencent.com>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      cb1d220d
  4. 30 Nov, 2021 17 commits
    • Paolo Bonzini's avatar
      KVM: fix avic_set_running for preemptable kernels · 7cfc5c65
      Paolo Bonzini authored
      avic_set_running() passes the current CPU to avic_vcpu_load(), albeit
      via vcpu->cpu rather than smp_processor_id().  If the thread is migrated
      while avic_set_running runs, the call to avic_vcpu_load() can use a stale
      value for the processor id.  Avoid this by blocking preemption over the
      entire execution of avic_set_running().
      Reported-by: default avatarSean Christopherson <seanjc@google.com>
      Fixes: 8221c137 ("svm: Manage vcpu load/unload when enable AVIC")
      Cc: stable@vger.kernel.org
      Reviewed-by: default avatarMaxim Levitsky <mlevitsk@redhat.com>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      7cfc5c65
    • Paolo Bonzini's avatar
      KVM: VMX: clear vmx_x86_ops.sync_pir_to_irr if APICv is disabled · e90e51d5
      Paolo Bonzini authored
      There is nothing to synchronize if APICv is disabled, since neither
      other vCPUs nor assigned devices can set PIR.ON.
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      e90e51d5
    • Paolo Bonzini's avatar
      KVM: SEV: accept signals in sev_lock_two_vms · c9d61dcb
      Paolo Bonzini authored
      Generally, kvm->lock is not taken for a long time, but
      sev_lock_two_vms is different: it takes vCPU locks
      inside, so userspace can hold it back just by calling
      a vCPU ioctl.  Play it safe and use mutex_lock_killable.
      
      Message-Id: <20211123005036.2954379-13-pbonzini@redhat.com>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      c9d61dcb
    • Paolo Bonzini's avatar
      KVM: SEV: do not take kvm->lock when destroying · 10a37929
      Paolo Bonzini authored
      Taking the lock is useless since there are no other references,
      and there are already accesses (e.g. to sev->enc_context_owner)
      that do not take it.  So get rid of it.
      Reviewed-by: default avatarSean Christopherson <seanjc@google.com>
      Message-Id: <20211123005036.2954379-12-pbonzini@redhat.com>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      10a37929
    • Paolo Bonzini's avatar
      KVM: SEV: Prohibit migration of a VM that has mirrors · 17d44a96
      Paolo Bonzini authored
      VMs that mirror an encryption context rely on the owner to keep the
      ASID allocated.  Performing a KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM
      would cause a dangling ASID:
      
      1. copy context from A to B (gets ref to A)
      2. move context from A to L (moves ASID from A to L)
      3. close L (releases ASID from L, B still references it)
      
      The right way to do the handoff instead is to create a fresh mirror VM
      on the destination first:
      
      1. copy context from A to B (gets ref to A)
      [later] 2. close B (releases ref to A)
      3. move context from A to L (moves ASID from A to L)
      4. copy context from L to M
      
      So, catch the situation by adding a count of how many VMs are
      mirroring this one's encryption context.
      
      Fixes: 0b020f5a ("KVM: SEV: Add support for SEV-ES intra host migration")
      Message-Id: <20211123005036.2954379-11-pbonzini@redhat.com>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      17d44a96
    • Paolo Bonzini's avatar
      KVM: SEV: Do COPY_ENC_CONTEXT_FROM with both VMs locked · bf42b02b
      Paolo Bonzini authored
      Now that we have a facility to lock two VMs with deadlock
      protection, use it for the creation of mirror VMs as well.  One of
      COPY_ENC_CONTEXT_FROM(dst, src) and COPY_ENC_CONTEXT_FROM(src, dst)
      would always fail, so the combination is nonsensical and it is okay to
      return -EBUSY if it is attempted.
      
      This sidesteps the question of what happens if a VM is
      MOVE_ENC_CONTEXT_FROM'd at the same time as it is
      COPY_ENC_CONTEXT_FROM'd: the locking prevents that from
      happening.
      
      Cc: Peter Gonda <pgonda@google.com>
      Cc: Sean Christopherson <seanjc@google.com>
      Reviewed-by: default avatarSean Christopherson <seanjc@google.com>
      Message-Id: <20211123005036.2954379-10-pbonzini@redhat.com>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      bf42b02b
    • Paolo Bonzini's avatar
      selftests: sev_migrate_tests: add tests for KVM_CAP_VM_COPY_ENC_CONTEXT_FROM · dc79c9f4
      Paolo Bonzini authored
      I am putting the tests in sev_migrate_tests because the failure conditions are
      very similar and some of the setup code can be reused, too.
      
      The tests cover both successful creation of a mirror VM, and error
      conditions.
      
      Cc: Peter Gonda <pgonda@google.com>
      Cc: Sean Christopherson <seanjc@google.com>
      Message-Id: <20211123005036.2954379-9-pbonzini@redhat.com>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      dc79c9f4
    • Paolo Bonzini's avatar
      KVM: SEV: move mirror status to destination of KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM · 642525e3
      Paolo Bonzini authored
      Allow intra-host migration of a mirror VM; the destination VM will be
      a mirror of the same ASID as the source.
      
      Fixes: b5663931 ("KVM: SEV: Add support for SEV intra host migration")
      Reviewed-by: default avatarSean Christopherson <seanjc@google.com>
      Message-Id: <20211123005036.2954379-8-pbonzini@redhat.com>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      642525e3
    • Paolo Bonzini's avatar
      KVM: SEV: initialize regions_list of a mirror VM · 2b347a38
      Paolo Bonzini authored
      This was broken before the introduction of KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM,
      but technically harmless because the region list was unused for a mirror
      VM.  However, it is untidy and it now causes a NULL pointer access when
      attempting to move the encryption context of a mirror VM.
      
      Fixes: 54526d1f ("KVM: x86: Support KVM VMs sharing SEV context")
      Message-Id: <20211123005036.2954379-7-pbonzini@redhat.com>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      2b347a38
    • Paolo Bonzini's avatar
      KVM: SEV: cleanup locking for KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM · 501b580c
      Paolo Bonzini authored
      Encapsulate the handling of the migration_in_progress flag for both VMs in
      two functions sev_lock_two_vms and sev_unlock_two_vms.  It does not matter
      if KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM locks the destination struct kvm a bit
      later, and this change 1) keeps the cleanup chain of labels smaller 2)
      makes it possible for KVM_CAP_VM_COPY_ENC_CONTEXT_FROM to reuse the logic.
      
      Cc: Peter Gonda <pgonda@google.com>
      Cc: Sean Christopherson <seanjc@google.com>
      Message-Id: <20211123005036.2954379-6-pbonzini@redhat.com>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      501b580c
    • Paolo Bonzini's avatar
      KVM: SEV: do not use list_replace_init on an empty list · 4674164f
      Paolo Bonzini authored
      list_replace_init cannot be used if the source is an empty list,
      because "new->next->prev = new" will overwrite "old->next":
      
      				new				old
      				prev = new, next = new		prev = old, next = old
      new->next = old->next		prev = new, next = old		prev = old, next = old
      new->next->prev = new		prev = new, next = old		prev = old, next = new
      new->prev = old->prev		prev = old, next = old		prev = old, next = old
      new->next->prev = new		prev = old, next = old		prev = new, next = new
      
      The desired outcome instead would be to leave both old and new the same
      as they were (two empty circular lists).  Use list_cut_before, which
      already has the necessary check and is documented to discard the
      previous contents of the list that will hold the result.
      
      Fixes: b5663931 ("KVM: SEV: Add support for SEV intra host migration")
      Reviewed-by: default avatarSean Christopherson <seanjc@google.com>
      Message-Id: <20211123005036.2954379-5-pbonzini@redhat.com>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      4674164f
    • Paolo Bonzini's avatar
      KVM: x86: Use a stable condition around all VT-d PI paths · 53b7ca1a
      Paolo Bonzini authored
      Currently, checks for whether VT-d PI can be used refer to the current
      status of the feature in the current vCPU; or they more or less pick
      vCPU 0 in case a specific vCPU is not available.
      
      However, these checks do not attempt to synchronize with changes to
      the IRTE.  In particular, there is no path that updates the IRTE when
      APICv is re-activated on vCPU 0; and there is no path to wakeup a CPU
      that has APICv disabled, if the wakeup occurs because of an IRTE
      that points to a posted interrupt.
      
      To fix this, always go through the VT-d PI path as long as there are
      assigned devices and APICv is available on both the host and the VM side.
      Since the relevant condition was copied over three times, take the hint
      and factor it into a separate function.
      Suggested-by: default avatarSean Christopherson <seanjc@google.com>
      Cc: stable@vger.kernel.org
      Reviewed-by: default avatarSean Christopherson <seanjc@google.com>
      Reviewed-by: default avatarMaxim Levitsky <mlevitsk@redhat.com>
      Reviewed-by: default avatarDavid Matlack <dmatlack@google.com>
      Message-Id: <20211123004311.2954158-5-pbonzini@redhat.com>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      53b7ca1a
    • Paolo Bonzini's avatar
      KVM: x86: check PIR even for vCPUs with disabled APICv · 37c4dbf3
      Paolo Bonzini authored
      The IRTE for an assigned device can trigger a POSTED_INTR_VECTOR even
      if APICv is disabled on the vCPU that receives it.  In that case, the
      interrupt will just cause a vmexit and leave the ON bit set together
      with the PIR bit corresponding to the interrupt.
      
      Right now, the interrupt would not be delivered until APICv is re-enabled.
      However, fixing this is just a matter of always doing the PIR->IRR
      synchronization, even if the vCPU has temporarily disabled APICv.
      
      This is not a problem for performance, or if anything it is an
      improvement.  First, in the common case where vcpu->arch.apicv_active is
      true, one fewer check has to be performed.  Second, static_call_cond will
      elide the function call if APICv is not present or disabled.  Finally,
      in the case for AMD hardware we can remove the sync_pir_to_irr callback:
      it is only needed for apic_has_interrupt_for_ppr, and that function
      already has a fallback for !APICv.
      
      Cc: stable@vger.kernel.org
      Co-developed-by: default avatarSean Christopherson <seanjc@google.com>
      Signed-off-by: default avatarSean Christopherson <seanjc@google.com>
      Reviewed-by: default avatarMaxim Levitsky <mlevitsk@redhat.com>
      Reviewed-by: default avatarDavid Matlack <dmatlack@google.com>
      Message-Id: <20211123004311.2954158-4-pbonzini@redhat.com>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      37c4dbf3
    • Paolo Bonzini's avatar
      KVM: VMX: prepare sync_pir_to_irr for running with APICv disabled · 7e1901f6
      Paolo Bonzini authored
      If APICv is disabled for this vCPU, assigned devices may still attempt to
      post interrupts.  In that case, we need to cancel the vmentry and deliver
      the interrupt with KVM_REQ_EVENT.  Extend the existing code that handles
      injection of L1 interrupts into L2 to cover this case as well.
      
      vmx_hwapic_irr_update is only called when APICv is active so it would be
      confusing to add a check for vcpu->arch.apicv_active in there.  Instead,
      just use vmx_set_rvi directly in vmx_sync_pir_to_irr.
      
      Cc: stable@vger.kernel.org
      Reviewed-by: default avatarMaxim Levitsky <mlevitsk@redhat.com>
      Reviewed-by: default avatarDavid Matlack <dmatlack@google.com>
      Reviewed-by: default avatarSean Christopherson <seanjc@google.com>
      Message-Id: <20211123004311.2954158-3-pbonzini@redhat.com>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      7e1901f6
    • Maciej S. Szmigiero's avatar
      KVM: selftests: page_table_test: fix calculation of guest_test_phys_mem · 81835ee1
      Maciej S. Szmigiero authored
      A kvm_page_table_test run with its default settings fails on VMX due to
      memory region add failure:
      > ==== Test Assertion Failure ====
      >  lib/kvm_util.c:952: ret == 0
      >  pid=10538 tid=10538 errno=17 - File exists
      >     1  0x00000000004057d1: vm_userspace_mem_region_add at kvm_util.c:947
      >     2  0x0000000000401ee9: pre_init_before_test at kvm_page_table_test.c:302
      >     3   (inlined by) run_test at kvm_page_table_test.c:374
      >     4  0x0000000000409754: for_each_guest_mode at guest_modes.c:53
      >     5  0x0000000000401860: main at kvm_page_table_test.c:500
      >     6  0x00007f82ae2d8554: ?? ??:0
      >     7  0x0000000000401894: _start at ??:?
      >  KVM_SET_USER_MEMORY_REGION IOCTL failed,
      >  rc: -1 errno: 17
      >  slot: 1 flags: 0x0
      >  guest_phys_addr: 0xc0000000 size: 0x40000000
      
      This is because the memory range that this test is trying to add
      (0x0c0000000 - 0x100000000) conflicts with LAPIC mapping at 0x0fee00000.
      
      Looking at the code it seems that guest_test_*phys*_mem variable gets
      mistakenly overwritten with guest_test_*virt*_mem while trying to adjust
      the former for alignment.
      With the correct variable adjusted this test runs successfully.
      Signed-off-by: default avatarMaciej S. Szmigiero <maciej.szmigiero@oracle.com>
      Message-Id: <52e487458c3172923549bbcf9dfccfbe6faea60b.1637940473.git.maciej.szmigiero@oracle.com>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      81835ee1
    • Sean Christopherson's avatar
      KVM: x86/mmu: Handle "default" period when selectively waking kthread · f47491d7
      Sean Christopherson authored
      Account for the '0' being a default, "let KVM choose" period, when
      determining whether or not the recovery worker needs to be awakened in
      response to userspace reducing the period.  Failure to do so results in
      the worker not being awakened properly, e.g. when changing the period
      from '0' to any small-ish value.
      
      Fixes: 4dfe4f40 ("kvm: x86: mmu: Make NX huge page recovery period configurable")
      Cc: stable@vger.kernel.org
      Cc: Junaid Shahid <junaids@google.com>
      Signed-off-by: default avatarSean Christopherson <seanjc@google.com>
      Message-Id: <20211120015706.3830341-1-seanjc@google.com>
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      f47491d7
    • Paolo Bonzini's avatar
      KVM: MMU: shadow nested paging does not have PKU · 28f091bc
      Paolo Bonzini authored
      Initialize the mask for PKU permissions as if CR4.PKE=0, avoiding
      incorrect interpretations of the nested hypervisor's page tables.
      
      Cc: stable@vger.kernel.org
      Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      28f091bc