Commit 022620ee authored by Robin Murphy's avatar Robin Murphy Committed by Catalin Marinas

arm64: Implement array_index_mask_nospec()

Provide an optimised, assembly implementation of array_index_mask_nospec()
for arm64 so that the compiler is not in a position to transform the code
in ways which affect its ability to inhibit speculation (e.g. by introducing
conditional branches).

This is similar to the sequence used by x86, modulo architectural differences
in the carry/borrow flags.
Reviewed-by: default avatarMark Rutland <mark.rutland@arm.com>
Signed-off-by: default avatarRobin Murphy <robin.murphy@arm.com>
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
parent 669474e7
...@@ -41,6 +41,27 @@ ...@@ -41,6 +41,27 @@
#define dma_rmb() dmb(oshld) #define dma_rmb() dmb(oshld)
#define dma_wmb() dmb(oshst) #define dma_wmb() dmb(oshst)
/*
* Generate a mask for array_index__nospec() that is ~0UL when 0 <= idx < sz
* and 0 otherwise.
*/
#define array_index_mask_nospec array_index_mask_nospec
static inline unsigned long array_index_mask_nospec(unsigned long idx,
unsigned long sz)
{
unsigned long mask;
asm volatile(
" cmp %1, %2\n"
" sbc %0, xzr, xzr\n"
: "=r" (mask)
: "r" (idx), "Ir" (sz)
: "cc");
csdb();
return mask;
}
#define __smp_mb() dmb(ish) #define __smp_mb() dmb(ish)
#define __smp_rmb() dmb(ishld) #define __smp_rmb() dmb(ishld)
#define __smp_wmb() dmb(ishst) #define __smp_wmb() dmb(ishst)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment