- 01 Oct, 2021 11 commits
-
-
Hengqi Chen authored
A bunch of BPF maps do not support specifying BTF types for key and value. This is non-uniform and inconvenient[0]. Currently, libbpf uses a retry logic which removes BTF type IDs when BPF map creation failed. Instead of retrying, this commit recognizes those specialized maps and removes BTF type IDs when creating BPF map. [0] Closes: https://github.com/libbpf/libbpf/issues/355Signed-off-by:
Hengqi Chen <hengqi.chen@gmail.com> Signed-off-by:
Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20210930161456.3444544-2-hengqi.chen@gmail.com
-
Johan Almbladh authored
This patch adds a set of tests of BPF_LDX_MEM where both operand registers are the same register. Mainly testing 32-bit JITs that may load a 64-bit value in two 32-bit loads, and must not overwrite the address register. Signed-off-by:
Johan Almbladh <johan.almbladh@anyfinetworks.com> Signed-off-by:
Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20211001130348.3670534-11-johan.almbladh@anyfinetworks.com
-
Johan Almbladh authored
This patch adds a tests of ALU32 and ALU64 LSH/RSH/ARSH operations for the case when the two operands are the same register. Mainly intended to test JITs that implement ALU64 shifts with 32-bit CPU instructions. Also renamed related helper functions for consistency with the new tests. Signed-off-by:
-
Johan Almbladh authored
This patch adds tests of all register combinations for BPF_ATOMIC operations on both BPF_W and BPF_DW sizes. Signed-off-by:
-
Johan Almbladh authored
This patch replaces the current register combination test with new exhaustive tests. Before, only a subset of register combinations was tested for ALU64 DIV. Now, all combinatons of operand registers are tested, including the case when they are the same, and for all ALU32 and ALU64 operations. Signed-off-by:
-
Johan Almbladh authored
This patch moves the ALU LSH/RSH/ARSH reference computations into the common reference value function. Also fix typo in constants so they now have the intended values. Signed-off-by:
-
Johan Almbladh authored
This patch expands the register-clobbering-during-function-call tests to cover more all ALU32/64 MUL, DIV and MOD operations and all ATOMIC operations. In short, if a JIT implements a complex operation with a call to an external function, it must make sure to save and restore all its caller-saved registers that may be clobbered by the call. Signed-off-by:
-
Johan Almbladh authored
This patch adds tests to check that the source register is preserved when zero-extending a 32-bit value. In particular, it checks that the source operand is not zero-extended in-place. Signed-off-by:
-
Johan Almbladh authored
This patch adds a series of test to verify the operation of BPF_ATOMIC with BPF_DW and BPF_W sizes, for all power-of-two magnitudes of the register value operand. Also fixes a confusing typo in the comment for a related test. Signed-off-by:
-
Johan Almbladh authored
This patch updates the existing tests of BPF_ATOMIC operations to verify that a 32-bit register operand is properly zero-extended. In particular, it checks the operation on archs that require 32-bit operands to be properly zero-/sign-extended or the result is undefined, e.g. MIPS64. Signed-off-by:
-
Johan Almbladh authored
This patch adds a series of tests to verify the behavior of BPF_LDX and BPF_STX with BPF_B//W sizes in isolation. In particular, it checks that BPF_LDX zero-extendeds the result, and that BPF_STX does not overwrite adjacent bytes in memory. BPF_ST and operations on BPF_DW size are deemed to be sufficiently tested by existing tests. Signed-off-by:
-
- 30 Sep, 2021 3 commits
-
-
Kev Jackson authored
Header DOC on include/net/xdp.h contained a few English grammer and spelling errors. Signed-off-by:
Kev Jackson <foamdino@gmail.com> Signed-off-by:
Jesper Dangaard Brouer <brouer@redhat.com> Link: https://lore.kernel.org/bpf/YVVaWmKqA8l9Tm4J@kev-VirtualBox
-
Po-Hsu Lin authored
There are several test cases in the bpf directory are still using exit 0 when they need to be skipped. Use kselftest framework skip code instead so it can help us to distinguish the return status. Criterion to filter out what should be fixed in bpf directory: grep -r "exit 0" -B1 | grep -i skip This change might cause some false-positives if people are running these test scripts directly and only checking their return codes, which will change from 0 to 4. However I think the impact should be small as most of our scripts here are already using this skip code. And there will be no such issue if running them with the kselftest framework. Signed-off-by:
Po-Hsu Lin <po-hsu.lin@canonical.com> Signed-off-by:
-
Kumar Kartikeya Dwivedi authored
When the loader indicates an internal error (result of a checked bpf system call), it returns the result in attr.test.retval. However, tests that rely on ASSERT_OK_PTR on NULL (returned from light skeleton) may miss that NULL denotes an error if errno is set to 0. This would result in skel pointer being NULL, while ASSERT_OK_PTR returning 1, leading to a SEGV on dereference of skel, because libbpf_get_error relies on the assumption that errno is always set in case of error for ptr == NULL. In particular, this was observed for the ksyms_module test. When executed using `./test_progs -t ksyms`, prior tests manipulated errno and the test didn't crash when it failed at ksyms_module load, while using `./test_progs -t ksyms_module` crashed due to errno being untouched. Fixes: 67234743 (libbpf: Generate loader program out of BPF ELF file.) Signed-off-by:
Kumar Kartikeya Dwivedi <memxor@gmail.com> Signed-off-by:
Alexei Starovoitov <ast@kernel.org> Link: https://lore.kernel.org/bpf/20210927145941.1383001-11-memxor@gmail.com
-
- 29 Sep, 2021 6 commits
-
-
Toke Høiland-Jørgensen authored
The previous patch to ignore STT_SECTION symbols only added the ignore condition in one of them. This fails if there's more than one map definition in the 'maps' section, because the subsequent modulus check will fail, resulting in error messages like: libbpf: elf: unable to determine legacy map definition size in ./xdpdump_xdp.o Fix this by also ignoring STT_SECTION in the first loop. Fixes: c3e8c44a ("libbpf: Ignore STT_SECTION symbols in 'maps' section") Signed-off-by:
Toke Høiland-Jørgensen <toke@redhat.com> Signed-off-by:
-
Lorenz Bauer authored
We have a unit test that invokes an XDP program with 1m different inputs, aka 1m BPF_PROG_RUN syscalls. We run this test concurrently with slight variations in how we generated the input. Since commit f23c4b39 ("bpf: Start using the BPF dispatcher in BPF_TEST_RUN") the unit test has slowed down significantly. Digging deeper reveals that the concurrent tests are serialised in the kernel on the XDP dispatcher. This is a global resource that is protected by a mutex, on which we contend. Fix this by not calling into the XDP dispatcher if we only want to perform a single run of the BPF program. See: https://lore.kernel.org/bpf/CACAyw9_y4QumOW35qpgTbLsJ532uGq-kVW-VESJzGyiZkypnvw@mail.gmail.com/Signed-off-by:
Lorenz Bauer <lmb@cloudflare.com> Signed-off-by:
-
Alexei Starovoitov authored
Align gen_loader data to 8 byte boundary to make sure union bpf_attr, bpf_insns and other structs are aligned. Signed-off-by:
-
Kumar Kartikeya Dwivedi authored
Cleanup code uses while (cpu++ < cpu_cnt) for closing fds, which means it starts iterating from 1 for closing fds. If the first fd is -1, it skips over it and closes garbage fds (typically zero) in the remaining array. This leads to test failures for future tests when they end up storing fd 0 (as the slot becomes free due to close(0)) in ldimm64's BTF fd, ending up trying to match module BTF id with vmlinux. This was observed as spurious CI failure for the ksym_module_libbpf and module_attach tests. The test ends up closing fd 0 and breaking libbpf's assumption that module BTF fd will always be > 0, which leads to the kernel thinking that we are pointing to a BTF ID in vmlinux BTF. Fixes: 025bd7c7 (selftests/bpf: Add test for bpf_get_branch_snapshot) Signed-off-by:
Song Liu <songliubraving@fb.com> Link: https://lore.kernel.org/bpf/20210927145941.1383001-12-memxor@gmail.com
-
Magnus Karlsson authored
Fix a build error with clang in __xp_alloc(): [...] net/xdp/xsk_buff_pool.c:465:15: error: variable 'xskb' is uninitialized when used here [-Werror,-Wuninitialized] xp_release(xskb); ^~~~ This is correctly detected by clang, but not gcc. In fact, the xp_release() statement should not be there at all in the refactored code, just remove it. Fixes: 94033cd8 ("xsk: Optimize for aligned case") Reported-by:Nathan Chancellor <nathan@kernel.org> Signed-off-by:
Magnus Karlsson <magnus.karlsson@intel.com> Signed-off-by:
Maciej Fijalkowski <maciej.fijalkowski@intel.com> Link: https://lore.kernel.org/bpf/20210929061403.8587-1-magnus.karlsson@gmail.com
-
Yonghong Song authored
clang build kernel failed the selftest probe_user. $ ./test_progs -t probe_user $ ... $ test_probe_user:PASS:get_kprobe_res 0 nsec $ test_probe_user:FAIL:check_kprobe_res wrong kprobe res from probe read: 0.0.0.0:0 $ #94 probe_user:FAIL The test attached to kernel function __sys_connect(). In net/socket.c, we have int __sys_connect(int fd, struct sockaddr __user *uservaddr, int addrlen) { ...... } ... SYSCALL_DEFINE3(connect, int, fd, struct sockaddr __user *, uservaddr, int, addrlen) { return __sys_connect(fd, uservaddr, addrlen); } The gcc compiler (8.5.0) does not inline __sys_connect() in syscall entry function. But latest clang trunk did the inlining. So the bpf program is not triggered. To make the test more reliable, let us kprobe the syscall entry function instead. Note that x86_64, arm64 and s390 have syscall wrappers and they have to be handled specially. Signed-off-by:Yonghong Song <yhs@fb.com> Signed-off-by:
-
- 28 Sep, 2021 20 commits
-
-
Alexei Starovoitov authored
Kees Cook says: ==================== Hi, In order to keep ahead of cases in the kernel where Control Flow Integrity (CFI) may trip over function call casts, enabling -Wcast-function-type is helpful. To that end, replace BPF_CAST_CALL() as it triggers warnings with this option and is now one of the last places in the kernel in need of fixing. Thanks, -Kees v2: - rebase to bpf-next - add acks v1: https://lore.kernel.org/lkml/20210927182700.2980499-1-keescook@chromium.org ==================== Signed-off-by:
-
Kees Cook authored
In order to keep ahead of cases in the kernel where Control Flow Integrity (CFI) may trip over function call casts, enabling -Wcast-function-type is helpful. To that end, BPF_CAST_CALL causes various warnings and is one of the last places in the kernel triggering this warning. For actual function calls, replace BPF_CAST_CALL() with a typedef, which captures the same details about the given function pointers. This change results in no object code difference. Signed-off-by:
Kees Cook <keescook@chromium.org> Signed-off-by:
Gustavo A. R. Silva <gustavoars@kernel.org> Link: https://github.com/KSPP/linux/issues/20 Link: https://lore.kernel.org/lkml/CAEf4Bzb46=-J5Fxc3mMZ8JQPtK1uoE0q6+g6WPz53Cvx=CBEhw@mail.gmail.com Link: https://lore.kernel.org/bpf/20210928230946.4062144-3-keescook@chromium.org
-
Kees Cook authored
In order to keep ahead of cases in the kernel where Control Flow Integrity (CFI) may trip over function call casts, enabling -Wcast-function-type is helpful. To that end, BPF_CAST_CALL causes various warnings and is one of the last places in the kernel triggering this warning. Most places using BPF_CAST_CALL actually just want a void * to perform math on. It's not actually performing a call, so just use a different helper to get the void *, by way of the new BPF_CALL_IMM() helper, which can clean up a common copy/paste idiom as well. This change results in no object code difference. Signed-off-by:
-
Yucong Sun authored
"?:" is a GNU C extension, some environment has warning flags for its use, or even prohibit it directly. This patch avoid triggering these problems by simply expand it to its full form, no functionality change. Signed-off-by:
Yucong Sun <fallentree@fb.com> Signed-off-by:
-
Alexei Starovoitov authored
Andrii Nakryiko says: ==================== Implement opt-in stricter BPF program section name (SEC()) handling logic. For a lot of supported ELF section names, enforce exact section name match with no arbitrary characters added at the end. See patch #9 for more details. To allow this, patches #2 through #4 clean up and preventively fix selftests, normalizing existing SEC() usage across multiple selftests. While at it, those patches also reduce the amount of remaining bpf_object__find_program_by_title() uses, which should be completely removed soon, given it's an API with ambiguous semantics and will be deprecated and eventually removed in libbpf 1.0. Patch #1 also introduces SEC("tc") as an alias for SEC("classifier"). "tc" is a better and less misleading name, so patch #3 replaces all classifier* uses with nice and short SEC("tc"). Last patch is also fixing "sk_lookup/" definition to not require and not allow extra "/blah" parts after it, which serve no meaning. All the other patches are gradual internal libbpf changes to: - allow this optional strict logic for ELF section name handling; - allow new use case (for now for "struct_ops", but that could be extended to, say, freplace definitions), in which it can be used stand-alone to specify just type (SEC("struct_ops")), or also accept extra parameters which can be utilized by libbpf to either get more data or double-check valid use (e.g., SEC("struct_ops/dctcp_init") to specify desired struct_ops operation that is supposed to be implemented); - get libbpf's internal logic ready to allow other libraries and applications to specify their custom handlers for ELF section name for BPF programs. All the pieces are in place, the only thing preventing making this as public libbpf API is reliance on internal type for specifying BPF program load attributes. The work is planned to revamp related low-level libbpf APIs, at which point it will be possible to just re-use such new types for coordination between libbpf and custom handlers. These changes are a part of libbpf 1.0 effort ([0]). They are also intended to be applied on top of the previous preparatory series [1], so currently CI will be failing to apply them to bpf-next until that patch set is landed. Once it is landed, kernel-patches daemon will automatically retest this patch set. [0] https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#stricter-and-more-uniform-bpf-program-section-name-sec-handling [1] https://patchwork.kernel.org/project/netdevbpf/list/?series=547675&state=* v3->v4: - replace SEC("classifier*") with SEC("tc") (Daniel); v2->v3: - applied acks, addressed most feedback, added comments to new flags (Dave); v1->v2: - rebase onto latest bpf-next and resolve merge conflicts w/ Dave's changes. ==================== Signed-off-by: -
Andrii Nakryiko authored
Update "sk_lookup/" definition to be a stand-alone type specifier, with backwards-compatible prefix match logic in non-libbpf-1.0 mode. Currently in selftests all the "sk_lookup/<whatever>" uses just use <whatever> for duplicated unique name encoding, which is redundant as BPF program's name (C function name) uniquely and descriptively identifies the intended use for such BPF programs. With libbpf's SEC_DEF("sk_lookup") definition updated, switch existing sk_lookup programs to use "unqualified" SEC("sk_lookup") section names, with no random text after it. Signed-off-by:Dave Marchevsky <davemarchevsky@fb.com> Link: https://lore.kernel.org/bpf/20210928161946.2512801-11-andrii@kernel.org
-
Andrii Nakryiko authored
Implement strict ELF section name handling for BPF programs. It utilizes `libbpf_set_strict_mode()` framework and adds new flag: LIBBPF_STRICT_SEC_NAME. If this flag is set, libbpf will enforce exact section name matching for a lot of program types that previously allowed just partial prefix match. E.g., if previously SEC("xdp_whatever_i_want") was allowed, now in strict mode only SEC("xdp") will be accepted, which makes SEC("") definitions cleaner and more structured. SEC() now won't be used as yet another way to uniquely encode BPF program identifier (for that C function name is better and is guaranteed to be unique within bpf_object). Now SEC() is strictly BPF program type and, depending on program type, extra load/attach parameter specification. Libbpf completely supports multiple BPF programs in the same ELF section, so multiple BPF programs of the same type/specification easily co-exist together within the same bpf_object scope. Additionally, a new (for now internal) convention is introduced: section name that can be a stand-alone exact BPF program type specificator, but also could have extra parameters after '/' delimiter. An example of such section is "struct_ops", which can be specified by itself, but also allows to specify the intended operation to be attached to, e.g., "struct_ops/dctcp_init". Note, that "struct_ops_some_op" is not allowed. Such section definition is specified as "struct_ops+". This change is part of libbpf 1.0 effort ([0], [1]). [0] Closes: https://github.com/libbpf/libbpf/issues/271 [1] https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#stricter-and-more-uniform-bpf-program-section-name-sec-handlingSigned-off-by: -
Andrii Nakryiko authored
Complete SEC() table refactoring towards unified form by rewriting BPF_APROG_SEC and BPF_EAPROG_SEC definitions with SEC_DEF(SEC_ATTACHABLE_OPT) (for optional expected_attach_type) and SEC_DEF(SEC_ATTACHABLE) (mandatory expected_attach_type), respectively. Drop BPF_APROG_SEC, BPF_EAPROG_SEC, and BPF_PROG_SEC_IMPL macros after that, leaving SEC_DEF() macro as the only one used. Signed-off-by:
-
Andrii Nakryiko authored
Refactor ELF section handler definitions table to use a set of flags and unified SEC_DEF() macro. This allows for more succinct and table-like set of definitions, and allows to more easily extend the logic without adding more verbosity (this is utilized in later patches in the series). This approach is also making libbpf-internal program pre-load callback not rely on bpf_sec_def definition, which demonstrates that future pluggable ELF section handlers will be able to achieve similar level of integration without libbpf having to expose extra types and APIs. For starters, update SEC_DEF() definitions and make them more succinct. Also convert BPF_PROG_SEC() and BPF_APROG_COMPAT() definitions to a common SEC_DEF() use. Signed-off-by:
-
Andrii Nakryiko authored
Move closer to not relying on bpf_sec_def internals that won't be part of public API, when pluggable SEC() handlers will be allowed. Drop pre-calculated prefix length, and in various helpers don't rely on this prefix length availability. Also minimize reliance on knowing bpf_sec_def's prefix for few places where section prefix shortcuts are supported (e.g., tp vs tracepoint, raw_tp vs raw_tracepoint). Given checking some string for having a given string-constant prefix is such a common operation and so annoying to be done with pure C code, add a small macro helper, str_has_pfx(), and reuse it throughout libbpf.c where prefix comparison is performed. With __builtin_constant_p() it's possible to have a convenient helper that checks some string for having a given prefix, where prefix is either string literal (or compile-time known string due to compiler optimization) or just a runtime string pointer, which is quite convenient and saves a lot of typing and string literal duplication. Signed-off-by:
-
Andrii Nakryiko authored
Refactor internals of libbpf to allow adding custom SEC() handling logic easily from outside of libbpf. To that effect, each SEC()-handling registration sets mandatory program type/expected attach type for a given prefix and can provide three callbacks called at different points of BPF program lifetime: - init callback for right after bpf_program is initialized and prog_type/expected_attach_type is set. This happens during bpf_object__open() step, close to the very end of constructing bpf_object, so all the libbpf APIs for querying and updating bpf_program properties should be available; - pre-load callback is called right before BPF_PROG_LOAD command is called in the kernel. This callbacks has ability to set both bpf_program properties, as well as program load attributes, overriding and augmenting the standard libbpf handling of them; - optional auto-attach callback, which makes a given SEC() handler support auto-attachment of a BPF program through bpf_program__attach() API and/or BPF skeletons <skel>__attach() method. Each callbacks gets a `long cookie` parameter passed in, which is specified during SEC() handling. This can be used by callbacks to lookup whatever additional information is necessary. This is not yet completely ready to be exposed to the outside world, mainly due to non-public nature of struct bpf_prog_load_params. Instead of making it part of public API, we'll wait until the planned low-level libbpf API improvements for BPF_PROG_LOAD and other typical bpf() syscall APIs, at which point we'll have a public, probably OPTS-based, way to fully specify BPF program load parameters, which will be used as an interface for custom pre-load callbacks. But this change itself is already a good first step to unify the BPF program hanling logic even within the libbpf itself. As one example, all the extra per-program type handling (sleepable bit, attach_btf_id resolution, unsetting optional expected attach type) is now more obvious and is gathered in one place. Signed-off-by:
-
Andrii Nakryiko authored
Normalize all the other non-conforming SEC() usages across all selftests. This is in preparation for libbpf to start to enforce stricter SEC() rules in libbpf 1.0 mode. Signed-off-by:
-
Andrii Nakryiko authored
Convert all SEC("classifier*") uses to a new and strict SEC("tc") section name. In reference_tracking selftests switch from ambiguous searching by program title (section name) to non-ambiguous searching by name in some selftests, getting closer to completely removing bpf_object__find_program_by_title(). Signed-off-by: -
Andrii Nakryiko authored
Convert almost all SEC("xdp_blah") uses to strict SEC("xdp") to comply with strict libbpf 1.0 logic of exact section name match for XDP program types. There is only one exception, which is only tested through iproute2 and defines multiple XDP programs within the same BPF object. Given iproute2 still works in non-strict libbpf mode and it doesn't have means to specify XDP programs by its name (not section name/title), leave that single file alone for now until iproute2 gains lookup by function/program name. Signed-off-by: -
Andrii Nakryiko authored
As argued in [0], add "tc" ELF section definition for SCHED_CLS BPF program type. "classifier" is a misleading terminology and should be migrated away from. [0] https://lore.kernel.org/bpf/270e27b1-e5be-5b1c-b343-51bd644d0747@iogearbox.net/Signed-off-by:
-
Johan Almbladh authored
This patch adds a tail call limit test where the program also emits a BPF_CALL to an external function prior to the tail call. Mainly testing that JITed programs preserve its internal register state, for example tail call count, across such external calls. Signed-off-by:
-
Johan Almbladh authored
This patch fixes an error in the tail call limit test that caused the test to fail on for x86-64 JIT. Previously, the register R0 was used to report the total number of tail calls made. However, after a tail call fall-through, the value of the R0 register is undefined. Now, all tail call error path tests instead use context state to store the count. Fixes: 874be05f ("bpf, tests: Add tail call test suite") Reported-by:
Paul Chaignon <paul@cilium.io> Reported-by:
Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by:
-
Johan Almbladh authored
This patch adds tests of the high 32 bits of 64-bit BPF_END conversions. It also adds a mirrored set of tests where the source bytes are reversed. The MSB of each byte is now set on the high word instead, possibly affecting sign-extension during conversion in a different way. Mainly for JIT testing. Signed-off-by:
-
Johan Almbladh authored
This patch expands the branch conversion test introduced by 66e5eb84 ("bpf, tests: Add branch conversion JIT test"). The test now includes a JMP with maximum eBPF offset. This triggers branch conversion for the 64-bit MIPS JIT. Additional variants are also added for cases when the branch is taken or not taken. Signed-off-by:
-
Johan Almbladh authored
This patch adds a set of tests for JMP and JMP32 operations where the branch decision is know at JIT time. Mainly testing JIT behaviour. Signed-off-by:
-
