• Peter Xu's avatar
    mm/gup: handle huge pmd for follow_pmd_mask() · 4418c522
    Peter Xu authored
    Replace pmd_trans_huge() with pmd_leaf() to also cover pmd_huge() as long
    as enabled.
    
    FOLL_TOUCH and FOLL_SPLIT_PMD only apply to THP, not yet huge.
    
    Since now follow_trans_huge_pmd() can process hugetlb pages, renaming it
    into follow_huge_pmd() to match what it does.  Move it into gup.c so not
    depend on CONFIG_THP.
    
    When at it, move the ctx->page_mask setup into follow_huge_pmd(), only set
    it when the page is valid.  It was not a bug to set it before even if GUP
    failed (page==NULL), because follow_page_mask() callers always ignores
    page_mask if so.  But doing so makes the code cleaner.
    
    [peterx@redhat.com: allow follow_pmd_mask() to take hugetlb tail pages]
      Link: https://lkml.kernel.org/r/20240403013249.1418299-3-peterx@redhat.com
    Link: https://lkml.kernel.org/r/20240327152332.950956-12-peterx@redhat.comSigned-off-by: default avatarPeter Xu <peterx@redhat.com>
    Reviewed-by: default avatarJason Gunthorpe <jgg@nvidia.com>
    Tested-by: default avatarRyan Roberts <ryan.roberts@arm.com>
    Cc: Andrea Arcangeli <aarcange@redhat.com>
    Cc: Andrew Jones <andrew.jones@linux.dev>
    Cc: Aneesh Kumar K.V (IBM) <aneesh.kumar@kernel.org>
    Cc: Axel Rasmussen <axelrasmussen@google.com>
    Cc: Christophe Leroy <christophe.leroy@csgroup.eu>
    Cc: Christoph Hellwig <hch@infradead.org>
    Cc: David Hildenbrand <david@redhat.com>
    Cc: James Houghton <jthoughton@google.com>
    Cc: John Hubbard <jhubbard@nvidia.com>
    Cc: Kirill A. Shutemov <kirill@shutemov.name>
    Cc: Lorenzo Stoakes <lstoakes@gmail.com>
    Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
    Cc: Michael Ellerman <mpe@ellerman.id.au>
    Cc: "Mike Rapoport (IBM)" <rppt@kernel.org>
    Cc: Muchun Song <muchun.song@linux.dev>
    Cc: Rik van Riel <riel@surriel.com>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Cc: Yang Shi <shy828301@gmail.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    4418c522
internal.h 43.3 KB