Commit 35282a2d authored by Brice Goglin's avatar Brice Goglin Committed by Linus Torvalds

migration: only migrate_prep() once per move_pages()

migrate_prep() is fairly expensive (72us on 16-core barcelona 1.9GHz).
Commit 3140a227 improved move_pages()
throughput by breaking it into chunks, but it also made migrate_prep() be
called once per chunk (every 128pages or so) instead of once per
move_pages().

This patch reverts to calling migrate_prep() only once per chunk as we did
before 2.6.29.  It is also a followup to commit
0aedadf9 ("mm: move migrate_prep out from
under mmap_sem").

This improves migration throughput on the above machine from 600MB/s to
750MB/s.
Signed-off-by: default avatarBrice Goglin <Brice.Goglin@inria.fr>
Acked-by: default avatarChristoph Lameter <cl@linux-foundation.org>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Nick Piggin <nickpiggin@yahoo.com.au>
Cc: Hugh Dickins <hugh.dickins@tiscali.co.uk>
Cc: Rik van Riel <riel@redhat.com>
Cc: Lee Schermerhorn <lee.schermerhorn@hp.com>
Reviewed-by: default avatarKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
parent 7f33d49a
...@@ -820,7 +820,6 @@ static int do_move_page_to_node_array(struct mm_struct *mm, ...@@ -820,7 +820,6 @@ static int do_move_page_to_node_array(struct mm_struct *mm,
struct page_to_node *pp; struct page_to_node *pp;
LIST_HEAD(pagelist); LIST_HEAD(pagelist);
migrate_prep();
down_read(&mm->mmap_sem); down_read(&mm->mmap_sem);
/* /*
...@@ -907,6 +906,9 @@ static int do_pages_move(struct mm_struct *mm, struct task_struct *task, ...@@ -907,6 +906,9 @@ static int do_pages_move(struct mm_struct *mm, struct task_struct *task,
pm = (struct page_to_node *)__get_free_page(GFP_KERNEL); pm = (struct page_to_node *)__get_free_page(GFP_KERNEL);
if (!pm) if (!pm)
goto out; goto out;
migrate_prep();
/* /*
* Store a chunk of page_to_node array in a page, * Store a chunk of page_to_node array in a page,
* but keep the last one as a marker * but keep the last one as a marker
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment