• Joey Gouly's avatar
    arm64: alternatives: mark patch_alternative() as `noinstr` · a2c0b0fb
    Joey Gouly authored
    The alternatives code must be `noinstr` such that it does not patch itself,
    as the cache invalidation is only performed after all the alternatives have
    been applied.
    
    Mark patch_alternative() as `noinstr`. Mark branch_insn_requires_update()
    and get_alt_insn() with `__always_inline` since they are both only called
    through patch_alternative().
    
    Booting a kernel in QEMU TCG with KCSAN=y and ARM64_USE_LSE_ATOMICS=y caused
    a boot hang:
    [    0.241121] CPU: All CPU(s) started at EL2
    
    The alternatives code was patching the atomics in __tsan_read4() from LL/SC
    atomics to LSE atomics.
    
    The following fragment is using LL/SC atomics in the .text section:
      | <__tsan_unaligned_read4+304>:     ldxr    x6, [x2]
      | <__tsan_unaligned_read4+308>:     add     x6, x6, x5
      | <__tsan_unaligned_read4+312>:     stxr    w7, x6, [x2]
      | <__tsan_unaligned_read4+316>:     cbnz    w7, <__tsan_unaligned_read4+304>
    
    This LL/SC atomic sequence was to be replaced with LSE atomics. However since
    the alternatives code was instrumentable, __tsan_read4() was being called after
    only the first instruction was replaced, which led to the following code in memory:
      | <__tsan_unaligned_read4+304>:     ldadd   x5, x6, [x2]
      | <__tsan_unaligned_read4+308>:     add     x6, x6, x5
      | <__tsan_unaligned_read4+312>:     stxr    w7, x6, [x2]
      | <__tsan_unaligned_read4+316>:     cbnz    w7, <__tsan_unaligned_read4+304>
    
    This caused an infinite loop as the `stxr` instruction never completed successfully,
    so `w7` was always 0.
    Signed-off-by: default avatarJoey Gouly <joey.gouly@arm.com>
    Cc: Mark Rutland <mark.rutland@arm.com>
    Cc: Catalin Marinas <catalin.marinas@arm.com>
    Cc: Will Deacon <will@kernel.org>
    Link: https://lore.kernel.org/r/20220405104733.11476-1-joey.gouly@arm.comSigned-off-by: default avatarWill Deacon <will@kernel.org>
    a2c0b0fb
alternative.c 6.75 KB