• Max Krasnyansky's avatar
    sched, cpuset: rework sched domains and CPU hotplug handling (v4) · cf417141
    Max Krasnyansky authored
    This is an updated version of my previous cpuset patch on top of
    the latest mainline git.
    The patch fixes CPU hotplug handling issues in the current cpusets code.
    Namely circular locking in rebuild_sched_domains() and unsafe access to
    the cpu_online_map in the cpuset cpu hotplug handler.
    
    This version includes changes suggested by Paul Jackson (naming, comments,
    style, etc). I also got rid of the separate workqueue thread because it is
    now safe to call get_online_cpus() from workqueue callbacks.
    
    Here are some more details:
    
    rebuild_sched_domains() is the only way to rebuild sched domains
    correctly based on the current cpuset settings. What this means
    is that we need to be able to call it from different contexts,
    like cpu hotplug for example.
    Also latest scheduler code in -tip now calls rebuild_sched_domains()
    directly from functions like arch_reinit_sched_domains().
    
    In order to support that properly we need to rework cpuset locking
    rules to avoid circular dependencies, which is what this patch does.
    New lock nesting rules are explained in the comments.
    We can now safely call rebuild_sched_domains() from virtually any
    context. The only requirement is that it needs to be called under
    get_online_cpus(). This allows cpu hotplug handlers and the scheduler
    to call rebuild_sched_domains() directly.
    The rest of the cpuset code now offloads sched domains rebuilds to
    a workqueue (async_rebuild_sched_domains()).
    
    This version of the patch addresses comments from the previous review.
    I fixed all miss-formated comments and trailing spaces.
    
    I also factored out the code that builds domain masks and split up CPU and
    memory hotplug handling. This was needed to simplify locking, to avoid unsafe
    access to the cpu_online_map from mem hotplug handler, and in general to make
    things cleaner.
    
    The patch passes moderate testing (building kernel with -j 16, creating &
    removing domains and bringing cpus off/online at the same time) on the
    quad-core2 based machine.
    
    It passes lockdep checks, even with preemptable RCU enabled.
    This time I also tested in with suspend/resume path and everything is working
    as expected.
    Signed-off-by: default avatarMax Krasnyansky <maxk@qualcomm.com>
    Acked-by: default avatarPaul Jackson <pj@sgi.com>
    Cc: menage@google.com
    Cc: a.p.zijlstra@chello.nl
    Cc: vegard.nossum@gmail.com
    Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
    cf417141
cpuset.c 70.6 KB