1. 30 Mar, 2021 5 commits
  2. 29 Mar, 2021 1 commit
  3. 28 Mar, 2021 1 commit
  4. 27 Mar, 2021 15 commits
    • Alexei Starovoitov's avatar
      Merge branch 'bpf: Support calling kernel function' · fddbf4b6
      Alexei Starovoitov authored
      Martin KaFai says:
      
      ====================
      
      This series adds support to allow bpf program calling kernel function.
      
      The use case included in this set is to allow bpf-tcp-cc to directly
      call some tcp-cc helper functions (e.g. "tcp_cong_avoid_ai()").  Those
      functions have already been used by some kernel tcp-cc implementations.
      
      This set will also allow the bpf-tcp-cc program to directly call the
      kernel tcp-cc implementation,  For example, a bpf_dctcp may only want to
      implement its own dctcp_cwnd_event() and reuse other dctcp_*() directly
      from the kernel tcp_dctcp.c instead of reimplementing (or
      copy-and-pasting) them.
      
      The tcp-cc kernel functions mentioned above will be white listed
      for the struct_ops bpf-tcp-cc programs to use in a later patch.
      The white listed functions are not bounded to a fixed ABI contract.
      Those functions have already been used by the existing kernel tcp-cc.
      If any of them has changed, both in-tree and out-of-tree kernel tcp-cc
      implementations have to be changed.  The same goes for the struct_ops
      bpf-tcp-cc programs which have to be adjusted accordingly.
      
      Please see individual patch for details.
      
      v2:
      - Patch 2 in v1 is removed.  No need to support extern func in kernel.
        Changed libbpf to adjust the .ksyms datasec for extern func
        in patch 11. (Andrii)
      - Name change: btf_check_func_arg_match() and btf_check_subprog_arg_match()
        in patch 2. (Andrii)
      - Always set unreliable on any error in patch 2 since it does not
        matter. (Andrii)
      - s/kern_func/kfunc/ and s/descriptor/desc/ in this set. (Andrii)
      - Remove some unnecessary changes in disasm.h and disasm.c
        in patch 3.  In particular, no need to change the function
        signature in bpf_insn_revmap_call_t.  Also, removed the changes
        in print_bpf_insn().
      - Fixed an issue in check_kfunc_call() when the calling kernel function
        returns a pointer in patch 3.  Added a selftest.
      - Adjusted the verifier selftests due to the changes in the verifier log
        in patch 3.
      - Fixed a comparison issue in kfunc_desc_cmp_by_imm() in patch 3. (Andrii)
      - Name change: is_ldimm64_insn(),
        new helper: is_call_insn() in patch 10 (Andrii)
      - Move btf_func_linkage() from btf.h to libbpf.c in patch 11. (Andrii)
      - Fixed the linker error when CONFIG_BPF_SYSCALL is not defined.
        Moved the check_kfunc_call from filter.c to test_run.c in patch 14.
        (kernel test robot)
      ====================
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      fddbf4b6
    • Martin KaFai Lau's avatar
      bpf: selftests: Add kfunc_call test · 7bd1590d
      Martin KaFai Lau authored
      This patch adds a few kernel function bpf_kfunc_call_test*() for the
      selftest's test_run purpose.  They will be allowed for tc_cls prog.
      
      The selftest calling the kernel function bpf_kfunc_call_test*()
      is also added in this patch.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Link: https://lore.kernel.org/bpf/20210325015252.1551395-1-kafai@fb.com
      7bd1590d
    • Martin KaFai Lau's avatar
      bpf: selftests: Bpf_cubic and bpf_dctcp calling kernel functions · 78e60bbb
      Martin KaFai Lau authored
      This patch removes the bpf implementation of tcp_slow_start()
      and tcp_cong_avoid_ai().  Instead, it directly uses the kernel
      implementation.
      
      It also replaces the bpf_cubic_undo_cwnd implementation by directly
      calling tcp_reno_undo_cwnd().  bpf_dctcp also directly calls
      tcp_reno_cong_avoid() instead.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20210325015246.1551062-1-kafai@fb.com
      78e60bbb
    • Martin KaFai Lau's avatar
      bpf: selftests: Rename bictcp to bpf_cubic · 39cd9e0f
      Martin KaFai Lau authored
      As a similar chanage in the kernel, this patch gives the proper
      name to the bpf cubic.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20210325015240.1550074-1-kafai@fb.com
      39cd9e0f
    • Martin KaFai Lau's avatar
      libbpf: Support extern kernel function · 5bd022ec
      Martin KaFai Lau authored
      This patch is to make libbpf able to handle the following extern
      kernel function declaration and do the needed relocations before
      loading the bpf program to the kernel.
      
      extern int foo(struct sock *) __attribute__((section(".ksyms")))
      
      In the collect extern phase, needed changes is made to
      bpf_object__collect_externs() and find_extern_btf_id() to collect
      extern function in ".ksyms" section.  The func in the BTF datasec also
      needs to be replaced by an int var.  The idea is similar to the existing
      handling in extern var.  In case the BTF may not have a var, a dummy ksym
      var is added at the beginning of bpf_object__collect_externs()
      if there is func under ksyms datasec.  It will also change the
      func linkage from extern to global which the kernel can support.
      It also assigns a param name if it does not have one.
      
      In the collect relo phase, it will record the kernel function
      call as RELO_EXTERN_FUNC.
      
      bpf_object__resolve_ksym_func_btf_id() is added to find the func
      btf_id of the running kernel.
      
      During actual relocation, it will patch the BPF_CALL instruction with
      src_reg = BPF_PSEUDO_FUNC_CALL and insn->imm set to the running
      kernel func's btf_id.
      
      The required LLVM patch: https://reviews.llvm.org/D93563Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Link: https://lore.kernel.org/bpf/20210325015234.1548923-1-kafai@fb.com
      5bd022ec
    • Martin KaFai Lau's avatar
      libbpf: Record extern sym relocation first · aa0b8d43
      Martin KaFai Lau authored
      This patch records the extern sym relocs first before recording
      subprog relocs.  The later patch will have relocs for extern
      kernel function call which is also using BPF_JMP | BPF_CALL.
      It will be easier to handle the extern symbols first in
      the later patch.
      
      is_call_insn() helper is added.  The existing is_ldimm64() helper
      is renamed to is_ldimm64_insn() for consistency.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20210325015227.1548623-1-kafai@fb.com
      aa0b8d43
    • Martin KaFai Lau's avatar
      libbpf: Rename RELO_EXTERN to RELO_EXTERN_VAR · 0c091e5c
      Martin KaFai Lau authored
      This patch renames RELO_EXTERN to RELO_EXTERN_VAR.
      It is to avoid the confusion with a later patch adding
      RELO_EXTERN_FUNC.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20210325015221.1547722-1-kafai@fb.com
      0c091e5c
    • Martin KaFai Lau's avatar
      libbpf: Refactor codes for finding btf id of a kernel symbol · 774e132e
      Martin KaFai Lau authored
      This patch refactors code, that finds kernel btf_id by kind
      and symbol name, to a new function find_ksym_btf_id().
      
      It also adds a new helper __btf_kind_str() to return
      a string by the numeric kind value.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20210325015214.1547069-1-kafai@fb.com
      774e132e
    • Martin KaFai Lau's avatar
      libbpf: Refactor bpf_object__resolve_ksyms_btf_id · 933d1aa3
      Martin KaFai Lau authored
      This patch refactors most of the logic from
      bpf_object__resolve_ksyms_btf_id() into a new function
      bpf_object__resolve_ksym_var_btf_id().
      It is to get ready for a later patch adding
      bpf_object__resolve_ksym_func_btf_id() which resolves
      a kernel function to the running kernel btf_id.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20210325015207.1546749-1-kafai@fb.com
      933d1aa3
    • Martin KaFai Lau's avatar
      bpf: tcp: Put some tcp cong functions in allowlist for bpf-tcp-cc · e78aea8b
      Martin KaFai Lau authored
      This patch puts some tcp cong helper functions, tcp_slow_start()
      and tcp_cong_avoid_ai(), into the allowlist for the bpf-tcp-cc
      program.
      
      A few tcp cc implementation functions are also put into the
      allowlist.  A potential use case is the bpf-tcp-cc implementation
      may only want to override a subset of a tcp_congestion_ops.  For others,
      the bpf-tcp-cc can directly call the kernel counter parts instead of
      re-implementing (or copy-and-pasting) them to the bpf program.
      
      They will only be available to the bpf-tcp-cc typed program.
      The allowlist functions are not bounded to a fixed ABI contract.
      When any of them has changed, the bpf-tcp-cc program has to be changed
      like any in-tree/out-of-tree kernel tcp-cc implementations do also.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20210325015201.1546345-1-kafai@fb.com
      e78aea8b
    • Martin KaFai Lau's avatar
      tcp: Rename bictcp function prefix to cubictcp · d22f6ad1
      Martin KaFai Lau authored
      The cubic functions in tcp_cubic.c are using the bictcp prefix as
      in tcp_bic.c.  This patch gives it the proper name cubictcp
      because the later patch will allow the bpf prog to directly
      call the cubictcp implementation.  Renaming them will avoid
      the name collision when trying to find the intended
      one to call during bpf prog load time.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Link: https://lore.kernel.org/bpf/20210325015155.1545532-1-kafai@fb.com
      d22f6ad1
    • Martin KaFai Lau's avatar
      bpf: Support kernel function call in x86-32 · 797b84f7
      Martin KaFai Lau authored
      This patch adds kernel function call support to the x86-32 bpf jit.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Link: https://lore.kernel.org/bpf/20210325015149.1545267-1-kafai@fb.com
      797b84f7
    • Martin KaFai Lau's avatar
      bpf: Support bpf program calling kernel function · e6ac2450
      Martin KaFai Lau authored
      This patch adds support to BPF verifier to allow bpf program calling
      kernel function directly.
      
      The use case included in this set is to allow bpf-tcp-cc to directly
      call some tcp-cc helper functions (e.g. "tcp_cong_avoid_ai()").  Those
      functions have already been used by some kernel tcp-cc implementations.
      
      This set will also allow the bpf-tcp-cc program to directly call the
      kernel tcp-cc implementation,  For example, a bpf_dctcp may only want to
      implement its own dctcp_cwnd_event() and reuse other dctcp_*() directly
      from the kernel tcp_dctcp.c instead of reimplementing (or
      copy-and-pasting) them.
      
      The tcp-cc kernel functions mentioned above will be white listed
      for the struct_ops bpf-tcp-cc programs to use in a later patch.
      The white listed functions are not bounded to a fixed ABI contract.
      Those functions have already been used by the existing kernel tcp-cc.
      If any of them has changed, both in-tree and out-of-tree kernel tcp-cc
      implementations have to be changed.  The same goes for the struct_ops
      bpf-tcp-cc programs which have to be adjusted accordingly.
      
      This patch is to make the required changes in the bpf verifier.
      
      First change is in btf.c, it adds a case in "btf_check_func_arg_match()".
      When the passed in "btf->kernel_btf == true", it means matching the
      verifier regs' states with a kernel function.  This will handle the
      PTR_TO_BTF_ID reg.  It also maps PTR_TO_SOCK_COMMON, PTR_TO_SOCKET,
      and PTR_TO_TCP_SOCK to its kernel's btf_id.
      
      In the later libbpf patch, the insn calling a kernel function will
      look like:
      
      insn->code == (BPF_JMP | BPF_CALL)
      insn->src_reg == BPF_PSEUDO_KFUNC_CALL /* <- new in this patch */
      insn->imm == func_btf_id /* btf_id of the running kernel */
      
      [ For the future calling function-in-kernel-module support, an array
        of module btf_fds can be passed at the load time and insn->off
        can be used to index into this array. ]
      
      At the early stage of verifier, the verifier will collect all kernel
      function calls into "struct bpf_kfunc_desc".  Those
      descriptors are stored in "prog->aux->kfunc_tab" and will
      be available to the JIT.  Since this "add" operation is similar
      to the current "add_subprog()" and looking for the same insn->code,
      they are done together in the new "add_subprog_and_kfunc()".
      
      In the "do_check()" stage, the new "check_kfunc_call()" is added
      to verify the kernel function call instruction:
      1. Ensure the kernel function can be used by a particular BPF_PROG_TYPE.
         A new bpf_verifier_ops "check_kfunc_call" is added to do that.
         The bpf-tcp-cc struct_ops program will implement this function in
         a later patch.
      2. Call "btf_check_kfunc_args_match()" to ensure the regs can be
         used as the args of a kernel function.
      3. Mark the regs' type, subreg_def, and zext_dst.
      
      At the later do_misc_fixups() stage, the new fixup_kfunc_call()
      will replace the insn->imm with the function address (relative
      to __bpf_call_base).  If needed, the jit can find the btf_func_model
      by calling the new bpf_jit_find_kfunc_model(prog, insn).
      With the imm set to the function address, "bpftool prog dump xlated"
      will be able to display the kernel function calls the same way as
      it displays other bpf helper calls.
      
      gpl_compatible program is required to call kernel function.
      
      This feature currently requires JIT.
      
      The verifier selftests are adjusted because of the changes in
      the verbose log in add_subprog_and_kfunc().
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Link: https://lore.kernel.org/bpf/20210325015142.1544736-1-kafai@fb.com
      e6ac2450
    • Martin KaFai Lau's avatar
      bpf: Refactor btf_check_func_arg_match · 34747c41
      Martin KaFai Lau authored
      This patch moved the subprog specific logic from
      btf_check_func_arg_match() to the new btf_check_subprog_arg_match().
      The core logic is left in btf_check_func_arg_match() which
      will be reused later to check the kernel function call.
      
      The "if (!btf_type_is_ptr(t))" is checked first to improve the
      indentation which will be useful for a later patch.
      
      Some of the "btf_kind_str[]" usages is replaced with the shortcut
      "btf_type_str(t)".
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Link: https://lore.kernel.org/bpf/20210325015136.1544504-1-kafai@fb.com
      34747c41
    • Martin KaFai Lau's avatar
      bpf: Simplify freeing logic in linfo and jited_linfo · e16301fb
      Martin KaFai Lau authored
      This patch simplifies the linfo freeing logic by combining
      "bpf_prog_free_jited_linfo()" and "bpf_prog_free_unused_jited_linfo()"
      into the new "bpf_prog_jit_attempt_done()".
      It is a prep work for the kernel function call support.  In a later
      patch, freeing the kernel function call descriptors will also
      be done in the "bpf_prog_jit_attempt_done()".
      
      "bpf_prog_free_linfo()" is removed since it is only called by
      "__bpf_prog_put_noref()".  The kvfree() are directly called
      instead.
      
      It also takes this chance to s/kcalloc/kvcalloc/ for the jited_linfo
      allocation.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Link: https://lore.kernel.org/bpf/20210325015130.1544323-1-kafai@fb.com
      e16301fb
  5. 26 Mar, 2021 11 commits
  6. 25 Mar, 2021 7 commits
    • Daniel Borkmann's avatar
      bpf: Undo ptr_to_map_key alu sanitation for now · 80847a71
      Daniel Borkmann authored
      Remove PTR_TO_MAP_KEY for the time being from being sanitized on pointer ALU
      through sanitize_ptr_alu() mainly for 3 reasons:
      
        1) It's currently unused and not available from unprivileged. However that by
           itself is not yet a strong reason to drop the code.
      
        2) Commit 69c087ba ("bpf: Add bpf_for_each_map_elem() helper") implemented
           the sanitation not fully correct in that unlike stack or map_value pointer
           it doesn't probe whether the access to the map key /after/ the simulated ALU
           operation is still in bounds. This means that the generated mask can truncate
           the offset in the non-speculative domain whereas it should only truncate in
           the speculative domain. The verifier should instead reject such program as
           we do for other types.
      
        3) Given the recent fixes from f232326f ("bpf: Prohibit alu ops for pointer
           types not defining ptr_limit"), 10d2bb2e ("bpf: Fix off-by-one for area
           size in creating mask to left"), b5871dca ("bpf: Simplify alu_limit masking
           for pointer arithmetic") as well as 1b1597e6 ("bpf: Add sanity check for
           upper ptr_limit") the code changed quite a bit and the merge in efd13b71
           broke the PTR_TO_MAP_KEY case due to an incorrect merge conflict.
      
      Remove the relevant pieces for the time being and we can rework the PTR_TO_MAP_KEY
      case once everything settles.
      
      Fixes: efd13b71 ("Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net")
      Fixes: 69c087ba ("bpf: Add bpf_for_each_map_elem() helper")
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      80847a71
    • David S. Miller's avatar
      Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next · 241949e4
      David S. Miller authored
      Alexei Starovoitov says:
      
      ====================
      pull-request: bpf-next 2021-03-24
      
      The following pull-request contains BPF updates for your *net-next* tree.
      
      We've added 37 non-merge commits during the last 15 day(s) which contain
      a total of 65 files changed, 3200 insertions(+), 738 deletions(-).
      
      The main changes are:
      
      1) Static linking of multiple BPF ELF files, from Andrii.
      
      2) Move drop error path to devmap for XDP_REDIRECT, from Lorenzo.
      
      3) Spelling fixes from various folks.
      ====================
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      241949e4
    • David S. Miller's avatar
    • Linus Torvalds's avatar
      Merge branch 'akpm' (patches from Andrew) · 00232240
      Linus Torvalds authored
      Merge misc fixes from Andrew Morton:
       "14 patches.
      
        Subsystems affected by this patch series: mm (hugetlb, kasan, gup,
        selftests, z3fold, kfence, memblock, and highmem), squashfs, ia64,
        gcov, and mailmap"
      
      * emailed patches from Andrew Morton <akpm@linux-foundation.org>:
        mailmap: update Andrey Konovalov's email address
        mm/highmem: fix CONFIG_DEBUG_KMAP_LOCAL_FORCE_MAP
        mm: memblock: fix section mismatch warning again
        kfence: make compatible with kmemleak
        gcov: fix clang-11+ support
        ia64: fix format strings for err_inject
        ia64: mca: allocate early mca with GFP_ATOMIC
        squashfs: fix xattr id and id lookup sanity checks
        squashfs: fix inode lookup sanity checks
        z3fold: prevent reclaim/free race for headless pages
        selftests/vm: fix out-of-tree build
        mm/mmu_notifiers: ensure range_end() is paired with range_start()
        kasan: fix per-page tags for non-page_alloc pages
        hugetlb_cgroup: fix imbalanced css_get and css_put pair for shared mappings
      00232240
    • Linus Torvalds's avatar
      Merge tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma · 2ba9bea2
      Linus Torvalds authored
      Pull rdma fixes from Jason Gunthorpe:
       "Not much going on, just some small bug fixes:
      
         - Typo causing a regression in mlx5 devx
      
         - Regression in the recent hns rework causing the HW to get out of
           sync
      
         - Long-standing cxgb4 adaptor crash when destroying cm ids"
      
      * tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma:
        RDMA/cxgb4: Fix adapter LE hash errors while destroying ipv6 listening server
        RDMA/hns: Fix bug during CMDQ initialization
        RDMA/mlx5: Fix typo in destroy_mkey inbox
      2ba9bea2
    • Linus Torvalds's avatar
      Merge tag 'mfd-fixes-5.12' of git://git.kernel.org/pub/scm/linux/kernel/git/lee/mfd · 58e4b9de
      Linus Torvalds authored
      Pull mfs fix from Lee Jones:
       "Unconstify editable placeholder structures"
      
      * tag 'mfd-fixes-5.12' of git://git.kernel.org/pub/scm/linux/kernel/git/lee/mfd:
        mfd: intel_quark_i2c_gpio: Revert "Constify static struct resources"
      58e4b9de
    • Linus Torvalds's avatar
      Merge tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux · 43f0b562
      Linus Torvalds authored
      Pull arm64 fixes from Will Deacon:
       "Minor fixes all over, ranging from typos to tests to errata
        workarounds:
      
         - Fix possible memory hotplug failure with KASLR
      
         - Fix FFR value in SVE kselftest
      
         - Fix backtraces reported in /proc/$pid/stack
      
         - Disable broken CnP implementation on NVIDIA Carmel
      
         - Typo fixes and ACPI documentation clarification
      
         - Fix some W=1 warnings"
      
      * tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux:
        arm64: kernel: disable CNP on Carmel
        arm64/process.c: fix Wmissing-prototypes build warnings
        kselftest/arm64: sve: Do not use non-canonical FFR register value
        arm64: mm: correct the inside linear map range during hotplug check
        arm64: kdump: update ppos when reading elfcorehdr
        arm64: cpuinfo: Fix a typo
        Documentation: arm64/acpi : clarify arm64 support of IBFT
        arm64: stacktrace: don't trace arch_stack_walk()
        arm64: csum: cast to the proper type
      43f0b562