• David Hildenbrand's avatar
    mm/rmap: rename hugepage_add* to hugetlb_add* · 9d5fafd5
    David Hildenbrand authored
    Patch series "mm/rmap: interface overhaul", v2.
    
    This series overhauls the rmap interface, to get rid of the "bool
    compound" / RMAP_COMPOUND parameter with the goal of making the interface
    less error prone, more future proof, and more natural to extend to
    "batching".  Also, this converts the interface to always consume
    folio+subpage, which speeds up operations on large folios.
    
    Further, this series adds PTE-batching variants for 4 rmap functions,
    whereby only folio_add_anon_rmap_ptes() is used for batching in this
    series when PTE-remapping a PMD-mapped THP.  folio_remove_rmap_ptes(),
    folio_try_dup_anon_rmap_ptes() and folio_dup_file_rmap_ptes() will soon
    come in handy[1,2].
    
    This series performs a lot of folio conversion along the way.  Most of the
    added LOC in the diff are only due to documentation.
    
    As we're moving to a pte/pmd interface where we clearly express the
    mapping granularity we are dealing with, we first get the remainder of
    hugetlb out of the way, as it is special and expected to remain special:
    it treats everything as a "single logical PTE" and only currently allows
    entire mappings.
    
    Even if we'd ever support partial mappings, I strongly assume the
    interface and implementation will still differ heavily: hopefull we can
    avoid working on subpages/subpage mapcounts completely and only add a
    "count" parameter for them to enable batching.
    
    New (extended) hugetlb interface that operates on entire folio:
     * hugetlb_add_new_anon_rmap() -> Already existed
     * hugetlb_add_anon_rmap() -> Already existed
     * hugetlb_try_dup_anon_rmap()
     * hugetlb_try_share_anon_rmap()
     * hugetlb_add_file_rmap()
     * hugetlb_remove_rmap()
    
    New "ordinary" interface for small folios / THP::
     * folio_add_new_anon_rmap() -> Already existed
     * folio_add_anon_rmap_[pte|ptes|pmd]()
     * folio_try_dup_anon_rmap_[pte|ptes|pmd]()
     * folio_try_share_anon_rmap_[pte|pmd]()
     * folio_add_file_rmap_[pte|ptes|pmd]()
     * folio_dup_file_rmap_[pte|ptes|pmd]()
     * folio_remove_rmap_[pte|ptes|pmd]()
    
    folio_add_new_anon_rmap() will always map at the largest granularity
    possible (currently, a single PMD to cover a PMD-sized THP).  Could be
    extended if ever required.
    
    In the future, we might want "_pud" variants and eventually "_pmds"
    variants for batching.
    
    I ran some simple microbenchmarks on an Intel(R) Xeon(R) Silver 4210R:
    measuring munmap(), fork(), cow, MADV_DONTNEED on each PTE ...  and PTE
    remapping PMD-mapped THPs on 1 GiB of memory.
    
    For small folios, there is barely a change (< 1% improvement for me).
    
    For PTE-mapped THP:
    * PTE-remapping a PMD-mapped THP is more than 10% faster.
    * fork() is more than 4% faster.
    * MADV_DONTNEED is 2% faster
    * COW when writing only a single byte on a COW-shared PTE is 1% faster
    * munmap() barely changes (< 1%).
    
    [1] https://lkml.kernel.org/r/20230810103332.3062143-1-ryan.roberts@arm.com
    [2] https://lkml.kernel.org/r/20231204105440.61448-1-ryan.roberts@arm.com
    
    
    This patch (of 40):
    
    Let's just call it "hugetlb_".
    
    Yes, it's all already inconsistent and confusing because we have a lot of
    "hugepage_" functions for legacy reasons.  But "hugetlb" cannot possibly
    be confused with transparent huge pages, and it matches "hugetlb.c" and
    "folio_test_hugetlb()".  So let's minimize confusion in rmap code.
    
    Link: https://lkml.kernel.org/r/20231220224504.646757-1-david@redhat.com
    Link: https://lkml.kernel.org/r/20231220224504.646757-2-david@redhat.comSigned-off-by: default avatarDavid Hildenbrand <david@redhat.com>
    Reviewed-by: default avatarMuchun Song <songmuchun@bytedance.com>
    Cc: Hugh Dickins <hughd@google.com>
    Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
    Cc: Muchun Song <muchun.song@linux.dev>
    Cc: Peter Xu <peterx@redhat.com>
    Cc: Ryan Roberts <ryan.roberts@arm.com>
    Cc: Yin Fengwei <fengwei.yin@intel.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    9d5fafd5
rmap.c 76.4 KB