• Linus Torvalds's avatar
    Reinstate ZERO_PAGE optimization in 'get_user_pages()' and fix XIP · 89f5b7da
    Linus Torvalds authored
    KAMEZAWA Hiroyuki and Oleg Nesterov point out that since the commit
    557ed1fa ("remove ZERO_PAGE") removed
    the ZERO_PAGE from the VM mappings, any users of get_user_pages() will
    generally now populate the VM with real empty pages needlessly.
    
    We used to get the ZERO_PAGE when we did the "handle_mm_fault()", but
    since fault handling no longer uses ZERO_PAGE for new anonymous pages,
    we now need to handle that special case in follow_page() instead.
    
    In particular, the removal of ZERO_PAGE effectively removed the core
    file writing optimization where we would skip writing pages that had not
    been populated at all, and increased memory pressure a lot by allocating
    all those useless newly zeroed pages.
    
    This reinstates the optimization by making the unmapped PTE case the
    same as for a non-existent page table, which already did this correctly.
    
    While at it, this also fixes the XIP case for follow_page(), where the
    caller could not differentiate between the case of a page that simply
    could not be used (because it had no "struct page" associated with it)
    and a page that just wasn't mapped.
    
    We do that by simply returning an error pointer for pages that could not
    be turned into a "struct page *".  The error is arbitrarily picked to be
    EFAULT, since that was what get_user_pages() already used for the
    equivalent IO-mapped page case.
    
    [ Also removed an impossible test for pte_offset_map_lock() failing:
      that's not how that function works ]
    Acked-by: default avatarOleg Nesterov <oleg@tv-sign.ru>
    Acked-by: default avatarNick Piggin <npiggin@suse.de>
    Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
    Cc: Hugh Dickins <hugh@veritas.com>
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Cc: Ingo Molnar <mingo@elte.hu>
    Cc: Roland McGrath <roland@redhat.com>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    89f5b7da
migrate.c 24.8 KB