• David Rientjes's avatar
    mm, thp: restore node-local hugepage allocations · 2f0799a0
    David Rientjes authored
    This is a full revert of ac5b2c18 ("mm: thp: relax __GFP_THISNODE for
    MADV_HUGEPAGE mappings") and a partial revert of 89c83fb5 ("mm, thp:
    consolidate THP gfp handling into alloc_hugepage_direct_gfpmask").
    
    By not setting __GFP_THISNODE, applications can allocate remote hugepages
    when the local node is fragmented or low on memory when either the thp
    defrag setting is "always" or the vma has been madvised with
    MADV_HUGEPAGE.
    
    Remote access to hugepages often has much higher latency than local pages
    of the native page size.  On Haswell, ac5b2c18 was shown to have a
    13.9% access regression after this commit for binaries that remap their
    text segment to be backed by transparent hugepages.
    
    The intent of ac5b2c18 is to address an issue where a local node is
    low on memory or fragmented such that a hugepage cannot be allocated.  In
    every scenario where this was described as a fix, there is abundant and
    unfragmented remote memory available to allocate from, even with a greater
    access latency.
    
    If remote memory is also low or fragmented, not setting __GFP_THISNODE was
    also measured on Haswell to have a 40% regression in allocation latency.
    
    Restore __GFP_THISNODE for thp allocations.
    
    Fixes: ac5b2c18 ("mm: thp: relax __GFP_THISNODE for MADV_HUGEPAGE mappings")
    Fixes: 89c83fb5 ("mm, thp: consolidate THP gfp handling into alloc_hugepage_direct_gfpmask")
    Cc: Andrea Arcangeli <aarcange@redhat.com>
    Cc: Mel Gorman <mgorman@techsingularity.net>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Cc: Michal Hocko <mhocko@kernel.org>
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarDavid Rientjes <rientjes@google.com>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    2f0799a0
huge_memory.c 81.8 KB