• Tejun Heo's avatar
    sched: replace migration_thread with cpu_stop · 969c7921
    Tejun Heo authored
    Currently migration_thread is serving three purposes - migration
    pusher, context to execute active_load_balance() and forced context
    switcher for expedited RCU synchronize_sched.  All three roles are
    hardcoded into migration_thread() and determining which job is
    scheduled is slightly messy.
    
    This patch kills migration_thread and replaces all three uses with
    cpu_stop.  The three different roles of migration_thread() are
    splitted into three separate cpu_stop callbacks -
    migration_cpu_stop(), active_load_balance_cpu_stop() and
    synchronize_sched_expedited_cpu_stop() - and each use case now simply
    asks cpu_stop to execute the callback as necessary.
    
    synchronize_sched_expedited() was implemented with private
    preallocated resources and custom multi-cpu queueing and waiting
    logic, both of which are provided by cpu_stop.
    synchronize_sched_expedited_count is made atomic and all other shared
    resources along with the mutex are dropped.
    
    synchronize_sched_expedited() also implemented a check to detect cases
    where not all the callback got executed on their assigned cpus and
    fall back to synchronize_sched().  If called with cpu hotplug blocked,
    cpu_stop already guarantees that and the condition cannot happen;
    otherwise, stop_machine() would break.  However, this patch preserves
    the paranoid check using a cpumask to record on which cpus the stopper
    ran so that it can serve as a bisection point if something actually
    goes wrong theree.
    
    Because the internal execution state is no longer visible,
    rcu_expedited_torture_stats() is removed.
    
    This patch also renames cpu_stop threads to from "stopper/%d" to
    "migration/%d".  The names of these threads ultimately don't matter
    and there's no reason to make unnecessary userland visible changes.
    
    With this patch applied, stop_machine() and sched now share the same
    resources.  stop_machine() is faster without wasting any resources and
    sched migration users are much cleaner.
    Signed-off-by: default avatarTejun Heo <tj@kernel.org>
    Acked-by: default avatarPeter Zijlstra <peterz@infradead.org>
    Cc: Ingo Molnar <mingo@elte.hu>
    Cc: Dipankar Sarma <dipankar@in.ibm.com>
    Cc: Josh Triplett <josh@freedesktop.org>
    Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
    Cc: Oleg Nesterov <oleg@redhat.com>
    Cc: Dimitri Sivanich <sivanich@sgi.com>
    969c7921
stop_machine.c 12.8 KB