• Shakeel Butt's avatar
    slab, slub: skip unnecessary kasan_cache_shutdown() · f9e13c0a
    Shakeel Butt authored
    The kasan quarantine is designed to delay freeing slab objects to catch
    use-after-free.  The quarantine can be large (several percent of machine
    memory size).  When kmem_caches are deleted related objects are flushed
    from the quarantine but this requires scanning the entire quarantine
    which can be very slow.  We have seen the kernel busily working on this
    while holding slab_mutex and badly affecting cache_reaper, slabinfo
    readers and memcg kmem cache creations.
    
    It can easily reproduced by following script:
    
    	yes . | head -1000000 | xargs stat > /dev/null
    	for i in `seq 1 10`; do
    		seq 500 | (cd /cg/memory && xargs mkdir)
    		seq 500 | xargs -I{} sh -c 'echo $BASHPID > \
    			/cg/memory/{}/tasks && exec stat .' > /dev/null
    		seq 500 | (cd /cg/memory && xargs rmdir)
    	done
    
    The busy stack:
        kasan_cache_shutdown
        shutdown_cache
        memcg_destroy_kmem_caches
        mem_cgroup_css_free
        css_free_rwork_fn
        process_one_work
        worker_thread
        kthread
        ret_from_fork
    
    This patch is based on the observation that if the kmem_cache to be
    destroyed is empty then there should not be any objects of this cache in
    the quarantine.
    
    Without the patch the script got stuck for couple of hours.  With the
    patch the script completed within a second.
    
    Link: http://lkml.kernel.org/r/20180327230603.54721-1-shakeelb@google.comSigned-off-by: default avatarShakeel Butt <shakeelb@google.com>
    Reviewed-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Acked-by: default avatarAndrey Ryabinin <aryabinin@virtuozzo.com>
    Acked-by: default avatarChristoph Lameter <cl@linux.com>
    Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
    Cc: Alexander Potapenko <glider@google.com>
    Cc: Greg Thelen <gthelen@google.com>
    Cc: Dmitry Vyukov <dvyukov@google.com>
    Cc: Pekka Enberg <penberg@kernel.org>
    Cc: David Rientjes <rientjes@google.com>
    Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    f9e13c0a
kasan.c 21.6 KB