• Ryan Roberts's avatar
    mm: swap: free_swap_and_cache_nr() as batched free_swap_and_cache() · a62fb92a
    Ryan Roberts authored
    Now that we no longer have a convenient flag in the cluster to determine
    if a folio is large, free_swap_and_cache() will take a reference and lock
    a large folio much more often, which could lead to contention and (e.g.)
    failure to split large folios, etc.
    
    Let's solve that problem by batch freeing swap and cache with a new
    function, free_swap_and_cache_nr(), to free a contiguous range of swap
    entries together.  This allows us to first drop a reference to each swap
    slot before we try to release the cache folio.  This means we only try to
    release the folio once, only taking the reference and lock once - much
    better than the previous 512 times for the 2M THP case.
    
    Contiguous swap entries are gathered in zap_pte_range() and
    madvise_free_pte_range() in a similar way to how present ptes are already
    gathered in zap_pte_range().
    
    While we are at it, let's simplify by converting the return type of both
    functions to void.  The return value was used only by zap_pte_range() to
    print a bad pte, and was ignored by everyone else, so the extra reporting
    wasn't exactly guaranteed.  We will still get the warning with most of the
    information from get_swap_device().  With the batch version, we wouldn't
    know which pte was bad anyway so could print the wrong one.
    
    [ryan.roberts@arm.com: fix a build warning on parisc]
      Link: https://lkml.kernel.org/r/20240409111840.3173122-1-ryan.roberts@arm.com
    Link: https://lkml.kernel.org/r/20240408183946.2991168-3-ryan.roberts@arm.comSigned-off-by: default avatarRyan Roberts <ryan.roberts@arm.com>
    Acked-by: default avatarDavid Hildenbrand <david@redhat.com>
    Cc: Barry Song <21cnbao@gmail.com>
    Cc: Barry Song <v-songbaohua@oppo.com>
    Cc: Chris Li <chrisl@kernel.org>
    Cc: Gao Xiang <xiang@kernel.org>
    Cc: "Huang, Ying" <ying.huang@intel.com>
    Cc: Kefeng Wang <wangkefeng.wang@huawei.com>
    Cc: Lance Yang <ioworker0@gmail.com>
    Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
    Cc: Michal Hocko <mhocko@suse.com>
    Cc: Yang Shi <shy828301@gmail.com>
    Cc: Yu Zhao <yuzhao@google.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    a62fb92a
memory.c 176 KB