• Peter Xu's avatar
    mm/hugetlb: move swap entry handling into vma lock when faulted · fcd48540
    Peter Xu authored
    In hugetlb_fault(), there used to have a special path to handle swap entry
    at the entrance using huge_pte_offset().  That's unsafe because
    huge_pte_offset() for a pmd sharable range can access freed pgtables if
    without any lock to protect the pgtable from being freed after pmd
    unshare.
    
    Here the simplest solution to make it safe is to move the swap handling to
    be after the vma lock being held.  We may need to take the fault mutex on
    either migration or hwpoison entries now (also the vma lock, but that's
    really needed), however neither of them is hot path.
    
    Note that the vma lock cannot be released in hugetlb_fault() when the
    migration entry is detected, because in migration_entry_wait_huge() the
    pgtable page will be used again (by taking the pgtable lock), so that also
    need to be protected by the vma lock.  Modify migration_entry_wait_huge()
    so that it must be called with vma read lock held, and properly release
    the lock in __migration_entry_wait_huge().
    
    Link: https://lkml.kernel.org/r/20221216155100.2043537-5-peterx@redhat.comSigned-off-by: default avatarPeter Xu <peterx@redhat.com>
    Reviewed-by: default avatarMike Kravetz <mike.kravetz@oracle.com>
    Reviewed-by: default avatarJohn Hubbard <jhubbard@nvidia.com>
    Cc: Andrea Arcangeli <aarcange@redhat.com>
    Cc: David Hildenbrand <david@redhat.com>
    Cc: James Houghton <jthoughton@google.com>
    Cc: Jann Horn <jannh@google.com>
    Cc: Miaohe Lin <linmiaohe@huawei.com>
    Cc: Muchun Song <songmuchun@bytedance.com>
    Cc: Nadav Amit <nadav.amit@gmail.com>
    Cc: Rik van Riel <riel@surriel.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    fcd48540
migrate.c 59.1 KB