Commit ccde85ba authored by Peter Zijlstra's avatar Peter Zijlstra Committed by Ingo Molnar

mm, locking: Fix up flush_tlb_pending() related merge in do_huge_pmd_numa_page()

Merge commit:

  040cca3a ("Merge branch 'linus' into locking/core, to resolve conflicts")

overlooked the fact that do_huge_pmd_numa_page() now does two TLB
flushes. Commit:

  8b1b436d ("mm, locking: Rework {set,clear,mm}_tlb_flush_pending()")

and commit:

  a9b80250 ("Revert "mm: numa: defer TLB flush for THP migration as long as possible"")

Both moved the TLB flush around but slightly different, the end result
being that what was one became two.

Clean this up.
Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: David S. Miller <davem@davemloft.net>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vineet Gupta <vgupta@synopsys.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
parent 040cca3a
...@@ -1410,7 +1410,6 @@ int do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t pmd) ...@@ -1410,7 +1410,6 @@ int do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t pmd)
unsigned long haddr = vmf->address & HPAGE_PMD_MASK; unsigned long haddr = vmf->address & HPAGE_PMD_MASK;
int page_nid = -1, this_nid = numa_node_id(); int page_nid = -1, this_nid = numa_node_id();
int target_nid, last_cpupid = -1; int target_nid, last_cpupid = -1;
bool need_flush = false;
bool page_locked; bool page_locked;
bool migrated = false; bool migrated = false;
bool was_writable; bool was_writable;
...@@ -1496,23 +1495,19 @@ int do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t pmd) ...@@ -1496,23 +1495,19 @@ int do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t pmd)
goto clear_pmdnuma; goto clear_pmdnuma;
} }
/*
* The page_table_lock above provides a memory barrier
* with change_protection_range.
*/
if (mm_tlb_flush_pending(vma->vm_mm))
flush_tlb_range(vma, haddr, haddr + HPAGE_PMD_SIZE);
/* /*
* Since we took the NUMA fault, we must have observed the !accessible * Since we took the NUMA fault, we must have observed the !accessible
* bit. Make sure all other CPUs agree with that, to avoid them * bit. Make sure all other CPUs agree with that, to avoid them
* modifying the page we're about to migrate. * modifying the page we're about to migrate.
* *
* Must be done under PTL such that we'll observe the relevant * Must be done under PTL such that we'll observe the relevant
* set_tlb_flush_pending(). * inc_tlb_flush_pending().
*
* We are not sure a pending tlb flush here is for a huge page
* mapping or not. Hence use the tlb range variant
*/ */
if (mm_tlb_flush_pending(vma->vm_mm)) if (mm_tlb_flush_pending(vma->vm_mm))
need_flush = true; flush_tlb_range(vma, haddr, haddr + HPAGE_PMD_SIZE);
/* /*
* Migrate the THP to the requested node, returns with page unlocked * Migrate the THP to the requested node, returns with page unlocked
...@@ -1520,13 +1515,6 @@ int do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t pmd) ...@@ -1520,13 +1515,6 @@ int do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t pmd)
*/ */
spin_unlock(vmf->ptl); spin_unlock(vmf->ptl);
/*
* We are not sure a pending tlb flush here is for a huge page
* mapping or not. Hence use the tlb range variant
*/
if (need_flush)
flush_tlb_range(vma, haddr, haddr + HPAGE_PMD_SIZE);
migrated = migrate_misplaced_transhuge_page(vma->vm_mm, vma, migrated = migrate_misplaced_transhuge_page(vma->vm_mm, vma,
vmf->pmd, pmd, vmf->address, page, target_nid); vmf->pmd, pmd, vmf->address, page, target_nid);
if (migrated) { if (migrated) {
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment