• Christoph Lameter's avatar
    SLUB: Move page->offset to kmem_cache_cpu->offset · b3fba8da
    Christoph Lameter authored
    We need the offset from the page struct during slab_alloc and slab_free. In
    both cases we also reference the cacheline of the kmem_cache_cpu structure.
    We can therefore move the offset field into the kmem_cache_cpu structure
    freeing up 16 bits in the page struct.
    
    Moving the offset allows an allocation from slab_alloc() without touching the
    page struct in the hot path.
    
    The only thing left in slab_free() that touches the page struct cacheline for
    per cpu freeing is the checking of SlabDebug(page). The next patch deals with
    that.
    
    Use the available 16 bits to broaden page->inuse. More than 64k objects per
    slab become possible and we can get rid of the checks for that limitation.
    
    No need anymore to shrink the order of slabs if we boot with 2M sized slabs
    (slub_min_order=9).
    
    No need anymore to switch off the offset calculation for very large slabs
    since the field in the kmem_cache_cpu structure is 32 bits and so the offset
    field can now handle slab sizes of up to 8GB.
    Signed-off-by: default avatarChristoph Lameter <clameter@sgi.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    b3fba8da
slub.c 89.6 KB