• Vlastimil Babka's avatar
    mm, slub: simplify kmem_cache_cpu and tid setup · 9b4bc85a
    Vlastimil Babka authored
    In slab_alloc_node() and do_slab_free() fastpaths we need to guarantee that
    our kmem_cache_cpu pointer is from the same cpu as the tid value. Currently
    that's done by reading the tid first using this_cpu_read(), then the
    kmem_cache_cpu pointer and verifying we read the same tid using the pointer and
    plain READ_ONCE().
    
    This can be simplified to just fetching kmem_cache_cpu pointer and then reading
    tid using the pointer. That guarantees they are from the same cpu. We don't
    need to read the tid using this_cpu_read() because the value will be validated
    by this_cpu_cmpxchg_double(), making sure we are on the correct cpu and the
    freelist didn't change by anyone preempting us since reading the tid.
    Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
    Acked-by: default avatarMel Gorman <mgorman@techsingularity.net>
    9b4bc85a
slub.c 145 KB