• Peter Xu's avatar
    mm/khugepaged: check again on anon uffd-wp during isolation · dd47ac42
    Peter Xu authored
    Khugepaged collapse an anonymous thp in two rounds of scans.  The 2nd
    round done in __collapse_huge_page_isolate() after
    hpage_collapse_scan_pmd(), during which all the locks will be released
    temporarily.  It means the pgtable can change during this phase before 2nd
    round starts.
    
    It's logically possible some ptes got wr-protected during this phase, and
    we can errornously collapse a thp without noticing some ptes are
    wr-protected by userfault.  e1e267c7 wanted to avoid it but it only
    did that for the 1st phase, not the 2nd phase.
    
    Since __collapse_huge_page_isolate() happens after a round of small page
    swapins, we don't need to worry on any !present ptes - if it existed
    khugepaged will already bail out.  So we only need to check present ptes
    with uffd-wp bit set there.
    
    This is something I found only but never had a reproducer, I thought it
    was one caused a bug in Muhammad's recent pagemap new ioctl work, but it
    turns out it's not the cause of that but an userspace bug.  However this
    seems to still be a real bug even with a very small race window, still
    worth to have it fixed and copy stable.
    
    Link: https://lkml.kernel.org/r/20230405155120.3608140-1-peterx@redhat.com
    Fixes: e1e267c7 ("khugepaged: skip collapse if uffd-wp detected")
    Signed-off-by: default avatarPeter Xu <peterx@redhat.com>
    Reviewed-by: default avatarDavid Hildenbrand <david@redhat.com>
    Reviewed-by: default avatarYang Shi <shy828301@gmail.com>
    Cc: Andrea Arcangeli <aarcange@redhat.com>
    Cc: Axel Rasmussen <axelrasmussen@google.com>
    Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
    Cc: Nadav Amit <nadav.amit@gmail.com>
    Cc: <stable@vger.kernel.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    dd47ac42
khugepaged.c 71.1 KB