• Joao Martins's avatar
    mm/memremap: add ZONE_DEVICE support for compound pages · c4386bd8
    Joao Martins authored
    Add a new @vmemmap_shift property for struct dev_pagemap which specifies
    that a devmap is composed of a set of compound pages of order
    @vmemmap_shift, instead of base pages.  When a compound page devmap is
    requested, all but the first page are initialised as tail pages instead
    of order-0 pages.
    
    For certain ZONE_DEVICE users like device-dax which have a fixed page
    size, this creates an opportunity to optimize GUP and GUP-fast walkers,
    treating it the same way as THP or hugetlb pages.
    
    Additionally, commit 7118fc29 ("hugetlb: address ref count racing in
    prep_compound_gigantic_page") removed set_page_count() because the
    setting of page ref count to zero was redundant.  devmap pages don't
    come from page allocator though and only head page refcount is used for
    compound pages, hence initialize tail page count to zero.
    
    Link: https://lkml.kernel.org/r/20211202204422.26777-5-joao.m.martins@oracle.comSigned-off-by: default avatarJoao Martins <joao.m.martins@oracle.com>
    Reviewed-by: default avatarDan Williams <dan.j.williams@intel.com>
    Cc: Christoph Hellwig <hch@lst.de>
    Cc: Dave Jiang <dave.jiang@intel.com>
    Cc: Jane Chu <jane.chu@oracle.com>
    Cc: Jason Gunthorpe <jgg@nvidia.com>
    Cc: Jason Gunthorpe <jgg@ziepe.ca>
    Cc: John Hubbard <jhubbard@nvidia.com>
    Cc: Jonathan Corbet <corbet@lwn.net>
    Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
    Cc: Mike Kravetz <mike.kravetz@oracle.com>
    Cc: Muchun Song <songmuchun@bytedance.com>
    Cc: Naoya Horiguchi <naoya.horiguchi@nec.com>
    Cc: Vishal Verma <vishal.l.verma@intel.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    c4386bd8
page_alloc.c 264 KB