• Vlastimil Babka's avatar
    mm, page_poison: use static key more efficiently · 8db26a3d
    Vlastimil Babka authored
    Commit 11c9c7ed ("mm/page_poison.c: replace bool variable with static
    key") changed page_poisoning_enabled() to a static key check.  However,
    the function is not inlined, so each check still involves a function call
    with overhead not eliminated when page poisoning is disabled.
    
    Analogically to how debug_pagealloc is handled, this patch converts
    page_poisoning_enabled() back to boolean check, and introduces
    page_poisoning_enabled_static() for fast paths.  Both functions are
    inlined.
    
    The function kernel_poison_pages() is also called unconditionally and does
    the static key check inside.  Remove it from there and put it to callers.
    Also split it to two functions kernel_poison_pages() and
    kernel_unpoison_pages() instead of the confusing bool parameter.
    
    Also optimize the check that enables page poisoning instead of
    debug_pagealloc for architectures without proper debug_pagealloc support.
    Move the check to init_mem_debugging_and_hardening() to enable a single
    static key instead of having two static branches in
    page_poisoning_enabled_static().
    
    Link: https://lkml.kernel.org/r/20201113104033.22907-3-vbabka@suse.czSigned-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
    Reviewed-by: default avatarDavid Hildenbrand <david@redhat.com>
    Cc: Mike Rapoport <rppt@linux.ibm.com>
    Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
    Cc: Alexander Potapenko <glider@google.com>
    Cc: Kees Cook <keescook@chromium.org>
    Cc: Laura Abbott <labbott@kernel.org>
    Cc: Mateusz Nosek <mateusznosek0@gmail.com>
    Cc: Michal Hocko <mhocko@kernel.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    8db26a3d
page_alloc.c 249 KB