• Vladimir Davydov's avatar
    list_lru: introduce per-memcg lists · 60d3fd32
    Vladimir Davydov authored
    There are several FS shrinkers, including super_block::s_shrink, that
    keep reclaimable objects in the list_lru structure.  Hence to turn them
    to memcg-aware shrinkers, it is enough to make list_lru per-memcg.
    
    This patch does the trick.  It adds an array of lru lists to the
    list_lru_node structure (per-node part of the list_lru), one for each
    kmem-active memcg, and dispatches every item addition or removal to the
    list corresponding to the memcg which the item is accounted to.  So now
    the list_lru structure is not just per node, but per node and per memcg.
    
    Not all list_lrus need this feature, so this patch also adds a new
    method, list_lru_init_memcg, which initializes a list_lru as memcg
    aware.  Otherwise (i.e.  if initialized with old list_lru_init), the
    list_lru won't have per memcg lists.
    
    Just like per memcg caches arrays, the arrays of per-memcg lists are
    indexed by memcg_cache_id, so we must grow them whenever
    memcg_nr_cache_ids is increased.  So we introduce a callback,
    memcg_update_all_list_lrus, invoked by memcg_alloc_cache_id if the id
    space is full.
    
    The locking is implemented in a manner similar to lruvecs, i.e.  we have
    one lock per node that protects all lists (both global and per cgroup) on
    the node.
    Signed-off-by: default avatarVladimir Davydov <vdavydov@parallels.com>
    Cc: Dave Chinner <david@fromorbit.com>
    Cc: Johannes Weiner <hannes@cmpxchg.org>
    Cc: Michal Hocko <mhocko@suse.cz>
    Cc: Greg Thelen <gthelen@google.com>
    Cc: Glauber Costa <glommer@gmail.com>
    Cc: Alexander Viro <viro@zeniv.linux.org.uk>
    Cc: Christoph Lameter <cl@linux.com>
    Cc: Pekka Enberg <penberg@kernel.org>
    Cc: David Rientjes <rientjes@google.com>
    Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
    Cc: Tejun Heo <tj@kernel.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    60d3fd32
list_lru.c 11 KB