• Mel Gorman's avatar
    mm: only drain per-cpu pagevecs once per pagevec usage · d9ed0d08
    Mel Gorman authored
    When a pagevec is initialised on the stack, it is generally used
    multiple times over a range of pages, looking up entries and then
    releasing them.  On each pagevec_release, the per-cpu deferred LRU
    pagevecs are drained on the grounds the page being released may be on
    those queues and the pages may be cache hot.  In many cases only the
    first drain is necessary as it's unlikely that the range of pages being
    walked is racing against LRU addition.  Even if there is such a race,
    the impact is marginal where as constantly redraining the lru pagevecs
    costs.
    
    This patch ensures that pagevec is only drained once in a given
    lifecycle without increasing the cache footprint of the pagevec
    structure.  Only sparsetruncate tiny is shown here as large files have
    many exceptional entries and calls pagecache_release less frequently.
    
    sparsetruncate (tiny)
                                  4.14.0-rc4             4.14.0-rc4
                            batchshadow-v1r1          onedrain-v1r1
    Min          Time      141.00 (   0.00%)      141.00 (   0.00%)
    1st-qrtle    Time      142.00 (   0.00%)      142.00 (   0.00%)
    2nd-qrtle    Time      142.00 (   0.00%)      142.00 (   0.00%)
    3rd-qrtle    Time      143.00 (   0.00%)      143.00 (   0.00%)
    Max-90%      Time      144.00 (   0.00%)      144.00 (   0.00%)
    Max-95%      Time      146.00 (   0.00%)      145.00 (   0.68%)
    Max-99%      Time      198.00 (   0.00%)      194.00 (   2.02%)
    Max          Time      254.00 (   0.00%)      208.00 (  18.11%)
    Amean        Time      145.12 (   0.00%)      144.30 (   0.56%)
    Stddev       Time       12.74 (   0.00%)        9.62 (  24.49%)
    Coeff        Time        8.78 (   0.00%)        6.67 (  24.06%)
    Best99%Amean Time      144.29 (   0.00%)      143.82 (   0.32%)
    Best95%Amean Time      142.68 (   0.00%)      142.31 (   0.26%)
    Best90%Amean Time      142.52 (   0.00%)      142.19 (   0.24%)
    Best75%Amean Time      142.26 (   0.00%)      141.98 (   0.20%)
    Best50%Amean Time      141.90 (   0.00%)      141.71 (   0.13%)
    Best25%Amean Time      141.80 (   0.00%)      141.43 (   0.26%)
    
    The impact on bonnie is marginal and within the noise because a
    significant percentage of the file being truncated has been reclaimed
    and consists of shadow entries which reduce the hotness of the
    pagevec_release path.
    
    Link: http://lkml.kernel.org/r/20171018075952.10627-5-mgorman@techsingularity.netSigned-off-by: default avatarMel Gorman <mgorman@techsingularity.net>
    Cc: Andi Kleen <ak@linux.intel.com>
    Cc: Dave Chinner <david@fromorbit.com>
    Cc: Dave Hansen <dave.hansen@intel.com>
    Cc: Jan Kara <jack@suse.cz>
    Cc: Johannes Weiner <hannes@cmpxchg.org>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    d9ed0d08
swap.c 28.1 KB