• Menglong Dong's avatar
    bpf, x86: allow function arguments up to 12 for TRACING · 473e3150
    Menglong Dong authored
    For now, the BPF program of type BPF_PROG_TYPE_TRACING can only be used
    on the kernel functions whose arguments count less than or equal to 6, if
    not considering '> 8 bytes' struct argument. This is not friendly at all,
    as too many functions have arguments count more than 6.
    
    According to the current kernel version, below is a statistics of the
    function arguments count:
    
    argument count | function count
    7              | 704
    8              | 270
    9              | 84
    10             | 47
    11             | 47
    12             | 27
    13             | 22
    14             | 5
    15             | 0
    16             | 1
    
    Therefore, let's enhance it by increasing the function arguments count
    allowed in arch_prepare_bpf_trampoline(), for now, only x86_64.
    
    For the case that we don't need to call origin function, which means
    without BPF_TRAMP_F_CALL_ORIG, we need only copy the function arguments
    that stored in the frame of the caller to current frame. The 7th and later
    arguments are stored in "$rbp + 0x18", and they will be copied to the
    stack area following where register values are saved.
    
    For the case with BPF_TRAMP_F_CALL_ORIG, we need prepare the arguments
    in stack before call origin function, which means we need alloc extra
    "8 * (arg_count - 6)" memory in the top of the stack. Note, there should
    not be any data be pushed to the stack before calling the origin function.
    So 'rbx' value will be stored on a stack position higher than where stack
    arguments are stored for BPF_TRAMP_F_CALL_ORIG.
    
    According to the research of Yonghong, struct members should be all in
    register or all on the stack. Meanwhile, the compiler will pass the
    argument on regs if the remaining regs can hold the argument. Therefore,
    we need save the arguments in order. Otherwise, disorder of the args can
    happen. For example:
    
      struct foo_struct {
          long a;
          int b;
      };
      int foo(char, char, char, char, char, struct foo_struct,
              char);
    
    the arg1-5,arg7 will be passed by regs, and arg6 will by stack. Therefore,
    we should save/restore the arguments in the same order with the
    declaration of foo(). And the args used as ctx in stack will be like this:
    
      reg_arg6   -- copy from regs
      stack_arg2 -- copy from stack
      stack_arg1
      reg_arg5   -- copy from regs
      reg_arg4
      reg_arg3
      reg_arg2
      reg_arg1
    
    We use EMIT3_off32() or EMIT4() for "lea" and "sub". The range of the
    imm in "lea" and "sub" is [-128, 127] if EMIT4() is used. Therefore,
    we use EMIT3_off32() instead if the imm out of the range.
    
    It works well for the FENTRY/FEXIT/MODIFY_RETURN.
    Signed-off-by: default avatarMenglong Dong <imagedong@tencent.com>
    Acked-by: default avatarYonghong Song <yhs@fb.com>
    Link: https://lore.kernel.org/r/20230713040738.1789742-3-imagedong@tencent.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
    473e3150
bpf_jit_comp.c 75.7 KB