• Sean Christopherson's avatar
    x86/speculation/l1tf: Exempt zeroed PTEs from inversion · f19f5c49
    Sean Christopherson authored
    It turns out that we should *not* invert all not-present mappings,
    because the all zeroes case is obviously special.
    
    clear_page() does not undergo the XOR logic to invert the address bits,
    i.e. PTE, PMD and PUD entries that have not been individually written
    will have val=0 and so will trigger __pte_needs_invert(). As a result,
    {pte,pmd,pud}_pfn() will return the wrong PFN value, i.e. all ones
    (adjusted by the max PFN mask) instead of zero. A zeroed entry is ok
    because the page at physical address 0 is reserved early in boot
    specifically to mitigate L1TF, so explicitly exempt them from the
    inversion when reading the PFN.
    
    Manifested as an unexpected mprotect(..., PROT_NONE) failure when called
    on a VMA that has VM_PFNMAP and was mmap'd to as something other than
    PROT_NONE but never used. mprotect() sends the PROT_NONE request down
    prot_none_walk(), which walks the PTEs to check the PFNs.
    prot_none_pte_entry() gets the bogus PFN from pte_pfn() and returns
    -EACCES because it thinks mprotect() is trying to adjust a high MMIO
    address.
    
    [ This is a very modified version of Sean's original patch, but all
      credit goes to Sean for doing this and also pointing out that
      sometimes the __pte_needs_invert() function only gets the protection
      bits, not the full eventual pte.  But zero remains special even in
      just protection bits, so that's ok.   - Linus ]
    
    Fixes: f22cc87f ("x86/speculation/l1tf: Invert all not present mappings")
    Signed-off-by: default avatarSean Christopherson <sean.j.christopherson@intel.com>
    Acked-by: default avatarAndi Kleen <ak@linux.intel.com>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Josh Poimboeuf <jpoimboe@redhat.com>
    Cc: Michal Hocko <mhocko@suse.com>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Cc: Dave Hansen <dave.hansen@intel.com>
    Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    f19f5c49
pgtable-invert.h 1.07 KB