• Vlastimil Babka's avatar
    mm/slub: handle bulk and single object freeing separately · 284f17ac
    Vlastimil Babka authored
    Currently we have a single function slab_free() handling both single
    object freeing and bulk freeing with necessary hooks, the latter case
    requiring slab_free_freelist_hook(). It should be however better to
    distinguish the two use cases for the following reasons:
    
    - code simpler to follow for the single object case
    
    - better code generation - although inlining should eliminate the
      slab_free_freelist_hook() for single object freeing in case no
      debugging options are enabled, it seems it's not perfect. When e.g.
      KASAN is enabled, we're imposing additional unnecessary overhead for
      single object freeing.
    
    - preparation to add percpu array caches in near future
    
    Therefore, simplify slab_free() for the single object case by dropping
    unnecessary parameters and calling only slab_free_hook() instead of
    slab_free_freelist_hook(). Rename the bulk variant to slab_free_bulk()
    and adjust callers accordingly.
    
    While at it, flip (and document) slab_free_hook() return value so that
    it returns true when the freeing can proceed, which matches the logic of
    slab_free_freelist_hook() and is not confusingly the opposite.
    
    Additionally we can simplify a bit by changing the tail parameter of
    do_slab_free() when freeing a single object - instead of NULL we can set
    it equal to head.
    
    bloat-o-meter shows small code reduction with a .config that has KASAN
    etc disabled:
    
    add/remove: 0/0 grow/shrink: 0/4 up/down: 0/-118 (-118)
    Function                                     old     new   delta
    kmem_cache_alloc_bulk                       1203    1196      -7
    kmem_cache_free                              861     835     -26
    __kmem_cache_free                            741     704     -37
    kmem_cache_free_bulk                         911     863     -48
    Reviewed-by: default avatarChengming Zhou <zhouchengming@bytedance.com>
    Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
    284f17ac
slub.c 177 KB