• Daniel Borkmann's avatar
    bpf, arm64: fix faulty emission of map access in tail calls · 3bfb04d1
    Daniel Borkmann authored
    
    [ Upstream commit d8b54110 ]
    
    Shubham was recently asking on netdev why in arm64 JIT we don't multiply
    the index for accessing the tail call map by 8. That led me into testing
    out arm64 JIT wrt tail calls and it turned out I got a NULL pointer
    dereference on the tail call.
    
    The buggy access is at:
    
      prog = array->ptrs[index];
      if (prog == NULL)
          goto out;
    
      [...]
      00000060:  d2800e0a  mov x10, #0x70 // #112
      00000064:  f86a682a  ldr x10, [x1,x10]
      00000068:  f862694b  ldr x11, [x10,x2]
      0000006c:  b40000ab  cbz x11, 0x00000080
      [...]
    
    The code triggering the crash is f862694b. x1 at the time contains the
    address of the bpf array, x10 offsetof(struct bpf_array, ptrs). Meaning,
    above we load the pointer to the program at map slot 0 into x10. x10
    can then be NULL if the slot is not occupied, which we later on try to
    access with a user given offset in x2 that is the map index.
    
    Fix this by emitting the following instead:
    
      [...]
      00000060:  d2800e0a  mov x10, #0x70 // #112
      00000064:  8b0a002a  add x10, x1, x10
      00000068:  d37df04b  lsl x11, x2, #3
      0000006c:  f86b694b  ldr x11, [x10,x11]
      00000070:  b40000ab  cbz x11, 0x00000084
      [...]
    
    This basically adds the offset to ptrs to the base address of the bpf
    array we got and we later on access the map with an index * 8 offset
    relative to that. The tail call map itself is basically one large area
    with meta data at the head followed by the array of prog pointers.
    This makes tail calls working again, tested on Cavium ThunderX ARMv8.
    
    Fixes: ddb55992 ("arm64: bpf: implement bpf_tail_call() helper")
    Reported-by: default avatarShubham Bansal <illusionist.neo@gmail.com>
    Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
    3bfb04d1
bpf_jit_comp.c 22.7 KB