Commit 68dc3ce3 authored by Hugh Dickins's avatar Hugh Dickins Committed by Linus Torvalds

[PATCH] mm: flush TLB when clearing young

Traditionally we've not flushed TLB after clearing the young/referenced bit,
it has seemed just a waste of time.  Russell King points out that on some
architectures, with the move from 2.4 mm sweeping to 2.6 rmap, this may be a
serious omission: very frequently referenced pages never re-marked young, and
the worst choices made for unmapping.

So, replace ptep_test_and_clear_young by ptep_clear_flush_young throughout
rmap.c.  Originally I'd imagined making some kind of TLB gather optimization,
but don't see what now: whether worth it rather depends on how common
cross-cpu flushes are, and whether global or not.

ppc and ppc64 have already found this issue, and worked around it by arranging
TLB flush from their ptep_test_and_clear_young: with the aid of pgtable rmap
pointers.  I'm hoping ptep_clear_flush_young will allow ppc and ppc64 to
remove that special code, but won't change them myself.

It's worth noting that it is Andrea's anon_vma rmap which makes the vma
available for ptep_clear_flush_young in page_referenced_one: anonmm and
pte_chains would both need an additional find_vma for that.
Signed-off-by: default avatarHugh Dickins <hugh@veritas.com>
Signed-off-by: default avatarAndrew Morton <akpm@osdl.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@osdl.org>
parent 3900b963
...@@ -226,7 +226,7 @@ static int page_referenced_one(struct page *page, ...@@ -226,7 +226,7 @@ static int page_referenced_one(struct page *page,
if (page_to_pfn(page) != pte_pfn(*pte)) if (page_to_pfn(page) != pte_pfn(*pte))
goto out_unmap; goto out_unmap;
if (ptep_test_and_clear_young(pte)) if (ptep_clear_flush_young(vma, address, pte))
referenced++; referenced++;
(*mapcount)--; (*mapcount)--;
...@@ -465,7 +465,7 @@ static int try_to_unmap_one(struct page *page, struct vm_area_struct *vma) ...@@ -465,7 +465,7 @@ static int try_to_unmap_one(struct page *page, struct vm_area_struct *vma)
* skipped over this mm) then we should reactivate it. * skipped over this mm) then we should reactivate it.
*/ */
if ((vma->vm_flags & (VM_LOCKED|VM_RESERVED)) || if ((vma->vm_flags & (VM_LOCKED|VM_RESERVED)) ||
ptep_test_and_clear_young(pte)) { ptep_clear_flush_young(vma, address, pte)) {
ret = SWAP_FAIL; ret = SWAP_FAIL;
goto out_unmap; goto out_unmap;
} }
...@@ -592,7 +592,7 @@ static int try_to_unmap_cluster(unsigned long cursor, ...@@ -592,7 +592,7 @@ static int try_to_unmap_cluster(unsigned long cursor,
if (PageReserved(page)) if (PageReserved(page))
continue; continue;
if (ptep_test_and_clear_young(pte)) if (ptep_clear_flush_young(vma, address, pte))
continue; continue;
/* Nuke the page table entry. */ /* Nuke the page table entry. */
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment