• Linus Torvalds's avatar
    x86: support user address masking instead of non-speculative conditional · 2865baf5
    Linus Torvalds authored
    The Spectre-v1 mitigations made "access_ok()" much more expensive, since
    it has to serialize execution with the test for a valid user address.
    
    All the normal user copy routines avoid this by just masking the user
    address with a data-dependent mask instead, but the fast
    "unsafe_user_read()" kind of patterms that were supposed to be a fast
    case got slowed down.
    
    This introduces a notion of using
    
    	src = masked_user_access_begin(src);
    
    to do the user address sanity using a data-dependent mask instead of the
    more traditional conditional
    
    	if (user_read_access_begin(src, len)) {
    
    model.
    
    This model only works for dense accesses that start at 'src' and on
    architectures that have a guard region that is guaranteed to fault in
    between the user space and the kernel space area.
    
    With this, the user access doesn't need to be manually checked, because
    a bad address is guaranteed to fault (by some architecture masking
    trick: on x86-64 this involves just turning an invalid user address into
    all ones, since we don't map the top of address space).
    
    This only converts a couple of examples for now.  Example x86-64 code
    generation for loading two words from user space:
    
            stac
            mov    %rax,%rcx
            sar    $0x3f,%rcx
            or     %rax,%rcx
            mov    (%rcx),%r13
            mov    0x8(%rcx),%r14
            clac
    
    where all the error handling and -EFAULT is now purely handled out of
    line by the exception path.
    
    Of course, if the micro-architecture does badly at 'clac' and 'stac',
    the above is still pitifully slow.  But at least we did as well as we
    could.
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    2865baf5
uaccess_64.h 5.47 KB