• Tim Chen's avatar
    mm/swap: add cache for swap slots allocation · 67afa38e
    Tim Chen authored
    We add per cpu caches for swap slots that can be allocated and freed
    quickly without the need to touch the swap info lock.
    
    Two separate caches are maintained for swap slots allocated and swap
    slots returned.  This is to allow the swap slots to be returned to the
    global pool in a batch so they will have a chance to be coaelesced with
    other slots in a cluster.  We do not reuse the slots that are returned
    right away, as it may increase fragmentation of the slots.
    
    The swap allocation cache is protected by a mutex as we may sleep when
    searching for empty slots in cache.  The swap free cache is protected by
    a spin lock as we cannot sleep in the free path.
    
    We refill the swap slots cache when we run out of slots, and we disable
    the swap slots cache and drain the slots if the global number of slots
    fall below a low watermark threshold.  We re-enable the cache agian when
    the slots available are above a high watermark.
    
    [ying.huang@intel.com: use raw_cpu_ptr over this_cpu_ptr for swap slots access]
    [tim.c.chen@linux.intel.com: add comments on locks in swap_slots.h]
      Link: http://lkml.kernel.org/r/20170118180327.GA24225@linux.intel.com
    Link: http://lkml.kernel.org/r/35de301a4eaa8daa2977de6e987f2c154385eb66.1484082593.git.tim.c.chen@linux.intel.comSigned-off-by: default avatarTim Chen <tim.c.chen@linux.intel.com>
    Signed-off-by: default avatar"Huang, Ying" <ying.huang@intel.com>
    Reviewed-by: default avatarMichal Hocko <mhocko@suse.com>
    Cc: Aaron Lu <aaron.lu@intel.com>
    Cc: Andi Kleen <ak@linux.intel.com>
    Cc: Andrea Arcangeli <aarcange@redhat.com>
    Cc: Christian Borntraeger <borntraeger@de.ibm.com>
    Cc: Dave Hansen <dave.hansen@intel.com>
    Cc: Hillf Danton <hillf.zj@alibaba-inc.com>
    Cc: Huang Ying <ying.huang@intel.com>
    Cc: Hugh Dickins <hughd@google.com>
    Cc: Johannes Weiner <hannes@cmpxchg.org>
    Cc: Jonathan Corbet <corbet@lwn.net> escreveu:
    Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
    Cc: Minchan Kim <minchan@kernel.org>
    Cc: Rik van Riel <riel@redhat.com>
    Cc: Shaohua Li <shli@kernel.org>
    Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    67afa38e
swap_slots.c 8.81 KB