• Peter Xu's avatar
    mm: Do early cow for pinned pages during fork() for ptes · 70e806e4
    Peter Xu authored
    This allows copy_pte_range() to do early cow if the pages were pinned on
    the source mm.
    
    Currently we don't have an accurate way to know whether a page is pinned
    or not.  The only thing we have is page_maybe_dma_pinned().  However
    that's good enough for now.  Especially, with the newly added
    mm->has_pinned flag to make sure we won't affect processes that never
    pinned any pages.
    
    It would be easier if we can do GFP_KERNEL allocation within
    copy_one_pte().  Unluckily, we can't because we're with the page table
    locks held for both the parent and child processes.  So the page
    allocation needs to be done outside copy_one_pte().
    
    Some trick is there in copy_present_pte(), majorly the wrprotect trick
    to block concurrent fast-gup.  Comments in the function should explain
    better in place.
    
    Oleg Nesterov reported a (probably harmless) bug during review that we
    didn't reset entry.val properly in copy_pte_range() so that potentially
    there's chance to call add_swap_count_continuation() multiple times on
    the same swp entry.  However that should be harmless since even if it
    happens, the same function (add_swap_count_continuation()) will return
    directly noticing that there're enough space for the swp counter.  So
    instead of a standalone stable patch, it is touched up in this patch
    directly.
    
    Link: https://lore.kernel.org/lkml/20200914143829.GA1424636@nvidia.com/Suggested-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: default avatarPeter Xu <peterx@redhat.com>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    70e806e4
memory.c 142 KB