• Will Deacon's avatar
    arm64: percpu: rewrite ll/sc loops in assembly · 1e6e57d9
    Will Deacon authored
    Writing the outer loop of an LL/SC sequence using do {...} while
    constructs potentially allows the compiler to hoist memory accesses
    between the STXR and the branch back to the LDXR. On CPUs that do not
    guarantee forward progress of LL/SC loops when faced with memory
    accesses to the same ERG (up to 2k) between the failed STXR and the
    branch back, we may end up livelocking.
    
    This patch avoids this issue in our percpu atomics by rewriting the
    outer loop as part of the LL/SC inline assembly block.
    
    Cc: <stable@vger.kernel.org>
    Fixes: f97fc810 ("arm64: percpu: Implement this_cpu operations")
    Reviewed-by: default avatarMark Rutland <mark.rutland@arm.com>
    Tested-by: default avatarMark Rutland <mark.rutland@arm.com>
    Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
    1e6e57d9
percpu.h 7.26 KB