• Martin Schwidefsky's avatar
    s390: improve wait logic of stop_machine · 38f2c691
    Martin Schwidefsky authored
    The stop_machine loop to advance the state machine and to wait for all
    affected CPUs to check-in calls cpu_relax_yield in a tight loop until
    the last missing CPUs acknowledged the state transition.
    
    On a virtual system where not all logical CPUs are backed by real CPUs
    all the time it can take a while for all CPUs to check-in. With the
    current definition of cpu_relax_yield a diagnose 0x44 is done which
    tells the hypervisor to schedule *some* other CPU. That can be any
    CPU and not necessarily one of the CPUs that need to run in order to
    advance the state machine. This can lead to a pretty bad diagnose 0x44
    storm until the last missing CPU finally checked-in.
    
    Replace the undirected cpu_relax_yield based on diagnose 0x44 with a
    directed yield. Each CPU in the wait loop will pick up the next CPU
    in the cpumask of stop_machine. The diagnose 0x9c is used to tell the
    hypervisor to run this next CPU instead of the current one. If there
    is only a limited number of real CPUs backing the virtual CPUs we
    end up with the real CPUs passed around in a round-robin fashion.
    
    [heiko.carstens@de.ibm.com]:
        Use cpumask_next_wrap as suggested by Peter Zijlstra.
    Signed-off-by: default avatarMartin Schwidefsky <schwidefsky@de.ibm.com>
    Acked-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Acked-by: default avatarThomas Gleixner <tglx@linutronix.de>
    Signed-off-by: default avatarHeiko Carstens <heiko.carstens@de.ibm.com>
    38f2c691
stop_machine.c 17.8 KB