- 24 Aug, 2020 2 commits
-
-
Martin KaFai Lau authored
This change is mostly from an internal patch and adapts it from sysctl config to the bpf_setsockopt setup. The bpf_prog can set the max delay ack by using bpf_setsockopt(TCP_BPF_DELACK_MAX). This max delay ack can be communicated to its peer through bpf header option. The receiving peer can then use this max delay ack and set a potentially lower rto by using bpf_setsockopt(TCP_BPF_RTO_MIN) which will be introduced in the next patch. Another later selftest patch will also use it like the above to show how to write and parse bpf tcp header option. Signed-off-by:
Martin KaFai Lau <kafai@fb.com> Signed-off-by:
Alexei Starovoitov <ast@kernel.org> Reviewed-by:
Eric Dumazet <edumazet@google.com> Acked-by:
John Fastabend <john.fastabend@gmail.com> Link: https://lore.kernel.org/bpf/20200820190021.2884000-1-kafai@fb.com
-
Martin KaFai Lau authored
The TCP_SAVE_SYN has both the network header and tcp header. The total length of the saved syn packet is currently stored in the first 4 bytes (u32) of an array and the actual packet data is stored after that. A later patch will add a bpf helper that allows to get the tcp header alone from the saved syn without the network header. It will be more convenient to have a direct offset to a specific header instead of re-parsing it. This requires to separately store the network hdrlen. The total header length (i.e. network + tcp) is still needed for the current usage in getsockopt. Although this total length can be obtained by looking into the tcphdr and then get the (th->doff << 2), this patch chooses to directly store the tcp hdrlen in the second four bytes of this newly created "struct saved_syn". By using a new struct, it can give a readable name to each individual header length. Signed-off-by:
-
- 21 Aug, 2020 17 commits
-
-
Andrii Nakryiko authored
Make libbpf logs follow similar pattern and provide more context like section name or program name, where appropriate. Also, add BPF_INSN_SZ constant and use it throughout to clean up code a little bit. This commit doesn't have any functional changes and just removes some code changes out of the way before bigger refactoring in libbpf internals. Signed-off-by:
Andrii Nakryiko <andriin@fb.com> Signed-off-by:
-
Andrii Nakryiko authored
Skip and don't log ELF sections that libbpf knows about and ignores during ELF processing. This allows to not unnecessarily log details about those ELF sections and cleans up libbpf debug log. Ignored sections include DWARF data, string table, empty .text section and few special (e.g., .llvm_addrsig) useless sections. With such ELF sections out of the way, log unrecognized ELF sections at pr_info level to increase visibility. Signed-off-by:
-
Andrii Nakryiko authored
__noinline is pretty frequently used, especially with BPF subprograms, so add them along the __always_inline, for user convenience and completeness. Signed-off-by:
-
Andrii Nakryiko authored
Factor out common ELF operations done throughout the libbpf. This simplifies usage across multiple places in libbpf, as well as hide error reporting from higher-level functions and make error logging more consistent. Signed-off-by:
-
Andrii Nakryiko authored
There is no need to re-build BPF object files if any of the sources of libbpf change. So record more precise dependency only on libbpf/bpf_*.h headers. This eliminates unnecessary re-builds. Signed-off-by:
-
Alexei Starovoitov authored
Lorenz Bauer says: ==================== We're currently building a control plane for our BPF socket dispatch work. As part of that, we have a need to create a copy of an existing sockhash, to allow us to change the keys. I previously proposed allowing privileged userspace to look up sockets, which doesn't work due to security concerns (see [1]). In follow up discussions during BPF office hours we identified bpf_iter as a possible solution: instead of accessing sockets from user space we can iterate the source sockhash, and insert the values into a new map. Enabling this requires two pieces: the ability to iterate sockmap and sockhash, as well as being able to call map_update_elem from BPF. This patch set implements the latter: it's now possible to update sockmap from BPF context. As a next step, we can implement bpf_iter for sockmap. === I've done some more fixups, and audited the safe contexts more thoroughly. As a result I'm removing CGROUP_SKB, SK_MSG and SK_SKB for now. Changes in v3: - Use CHECK as much as possible (Yonghong) - Reject ARG_PTR_TO_MAP_VALUE_OR_NULL for sockmap (Yonghong) - Remove CGROUP_SKB, SK_MSG, SK_SKB from safe contexts - Test that the verifier rejects update from unsafe context Changes in v2: - Fix warning in patch #2 (Jakub K) - Renamed override_map_arg_type (John) - Only allow updating sockmap from known safe contexts (John) - Use __s64 for sockmap updates from user space (Yonghong) - Various small test fixes around test macros and such (Yonghong) Thank your for your reviews! 1: https://lore.kernel.org/bpf/20200310174711.7490-1-lmb@cloudflare.com/ ==================== Signed-off-by:
-
Lorenz Bauer authored
Add a test which copies a socket from a sockmap into another sockmap or sockhash. This excercises bpf_map_update_elem support from BPF context. Compare the socket cookies from source and destination to ensure that the copy succeeded. Also check that the verifier rejects map_update from unsafe contexts. Signed-off-by:
Lorenz Bauer <lmb@cloudflare.com> Signed-off-by:
Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/bpf/20200821102948.21918-7-lmb@cloudflare.com
-
Lorenz Bauer authored
Allow calling bpf_map_update_elem on sockmap and sockhash from a BPF context. The synchronization required for this is a bit fiddly: we need to prevent the socket from changing its state while we add it to the sockmap, since we rely on getting a callback via sk_prot->unhash. However, we can't just lock_sock like in sock_map_sk_acquire because that might sleep. So instead we disable softirq processing and use bh_lock_sock to prevent further modification. Yet, this is still not enough. BPF can be called in contexts where the current CPU might have locked a socket. If the BPF can get a hold of such a socket, inserting it into a sockmap would lead to a deadlock. One straight forward example are sock_ops programs that have ctx->sk, but the same problem exists for kprobes, etc. We deal with this by allowing sockmap updates only from known safe contexts. Improper usage is rejected by the verifier. I've audited the enabled contexts to make sure they can't run in a locked context. It's possible that CGROUP_SKB and others are safe as well, but the auditing here is much more difficult. In any case, we can extend the safe contexts when the need arises. Signed-off-by:
-
Lorenz Bauer authored
The verifier assumes that map values are simple blobs of memory, and therefore treats ARG_PTR_TO_MAP_VALUE, etc. as such. However, there are map types where this isn't true. For example, sockmap and sockhash store sockets. In general this isn't a big problem: we can just write helpers that explicitly requests PTR_TO_SOCKET instead of ARG_PTR_TO_MAP_VALUE. The one exception are the standard map helpers like map_update_elem, map_lookup_elem, etc. Here it would be nice we could overload the function prototype for different kinds of maps. Unfortunately, this isn't entirely straight forward: We only know the type of the map once we have resolved meta->map_ptr in check_func_arg. This means we can't swap out the prototype in check_helper_call until we're half way through the function. Instead, modify check_func_arg to treat ARG_PTR_TO_MAP_VALUE to mean "the native type for the map" instead of "pointer to memory" for sockmap and sockhash. This means we don't have to modify the function prototype at all Signed-off-by:
-
Lorenz Bauer authored
Don't go via map->ops to call sock_map_update_elem, since we know what function to call in bpf_map_update_value. Since we currently don't allow calling map_update_elem from BPF context, we can remove ops->map_update_elem and rename the function to sock_map_update_elem_sys. Signed-off-by:
-
Lorenz Bauer authored
Merge the two very similar functions sock_map_update_elem and sock_hash_update_elem into one. Signed-off-by:
-
Lorenz Bauer authored
Initializing psock->sk_proto and other saved callbacks is only done in sk_psock_update_proto, after sk_psock_init has returned. The logic for this is difficult to follow, and needlessly complex. Instead, initialize psock->sk_proto whenever we allocate a new psock. Additionally, assert the following invariants: * The SK has no ULP: ULP does it's own finagling of sk->sk_prot * sk_user_data is unused: we need it to store sk_psock Protect our access to sk_user_data with sk_callback_lock, which is what other users like reuseport arrays, etc. do. The result is that an sk_psock is always fully initialized, and that psock->sk_proto is always the "original" struct proto. The latter allows us to use psock->sk_proto when initializing IPv6 TCP / UDP callbacks for sockmap. Signed-off-by:
-
Andrii Nakryiko authored
Add a set of APIs to perf_buffer manage to allow applications to integrate perf buffer polling into existing epoll-based infrastructure. One example is applications using libevent already and wanting to plug perf_buffer polling, instead of relying on perf_buffer__poll() and waste an extra thread to do it. But perf_buffer is still extremely useful to set up and consume perf buffer rings even for such use cases. So to accomodate such new use cases, add three new APIs: - perf_buffer__buffer_cnt() returns number of per-CPU buffers maintained by given instance of perf_buffer manager; - perf_buffer__buffer_fd() returns FD of perf_event corresponding to a specified per-CPU buffer; this FD is then polled independently; - perf_buffer__consume_buffer() consumes data from single per-CPU buffer, identified by its slot index. To support a simpler, but less efficient, way to integrate perf_buffer into external polling logic, also expose underlying epoll FD through perf_buffer__epoll_fd() API. It will need to be followed by perf_buffer__poll(), wasting extra syscall, or perf_buffer__consume(), wasting CPU to iterate buffers with no data. But could be simpler and more convenient for some cases. These APIs allow for great flexiblity, but do not sacrifice general usability of perf_buffer. Also exercise and check new APIs in perf_buffer selftest. Signed-off-by:Alan Maguire <alan.maguire@oracle.com> Link: https://lore.kernel.org/bpf/20200821165927.849538-1-andriin@fb.com
-
Alexei Starovoitov authored
Yonghong Song says: ==================== "link" has been an important concept for bpf ecosystem to connect bpf program with other properties. Currently, the information related information can be queried from userspace through bpf command BPF_LINK_GET_NEXT_ID, BPF_LINK_GET_FD_BY_ID and BPF_OBJ_GET_INFO_BY_FD. The information is also available by "cating" /proc/<pid>/fdinfo/<link_fd>. Raw_tracepoint, tracing, cgroup, netns and xdp links are already supported in the kernel and bpftool. This patch added support for bpf iterator. Patch #1 added generic support for link querying interface. Patch #2 implemented callback functions for map element bpf iterators. Patch #3 added bpftool support. Changelogs: v3 -> v4: . return target specific link_info even if target_name buffer is empty. (Andrii) v2 -> v3: . remove extra '\t' when fdinfo prints map_id to make parsing consistent. (Andrii) v1 -> v2: . fix checkpatch.pl warnings. (Jakub) ==================== Acked-by: -
Yonghong Song authored
The link query for bpf iterators is implemented. Besides being shown to the user what bpf iterator the link represents, the target_name is also used to filter out what additional information should be printed out, e.g., whether map_id should be shown or not. The following is an example of bpf_iter link dump, plain output or pretty output. $ bpftool link show 11: iter prog 59 target_name task pids test_progs(1749) 34: iter prog 173 target_name bpf_map_elem map_id 127 pids test_progs_1(1753) $ bpftool -p link show [{ "id": 11, "type": "iter", "prog_id": 59, "target_name": "task", "pids": [{ "pid": 1749, "comm": "test_progs" } ] },{ "id": 34, "type": "iter", "prog_id": 173, "target_name": "bpf_map_elem", "map_id": 127, "pids": [{ "pid": 1753, "comm": "test_progs_1" } ] } ] Signed-off-by: -
Yonghong Song authored
For bpf_map_elem and bpf_sk_local_storage bpf iterators, additional map_id should be shown for fdinfo and userspace query. For example, the following is for a bpf_map_elem iterator. $ cat /proc/1753/fdinfo/9 pos: 0 flags: 02000000 mnt_id: 14 link_type: iter link_id: 34 prog_tag: 104be6d3fe45e6aa prog_id: 173 target_name: bpf_map_elem map_id: 127 Signed-off-by:
-
Yonghong Song authored
This patch implemented bpf_link callback functions show_fdinfo and fill_link_info to support link_query interface. The general interface for show_fdinfo and fill_link_info will print/fill the target_name. Each targets can register show_fdinfo and fill_link_info callbacks to print/fill more target specific information. For example, the below is a fdinfo result for a bpf task iterator. $ cat /proc/1749/fdinfo/7 pos: 0 flags: 02000000 mnt_id: 14 link_type: iter link_id: 11 prog_tag: 990e1f8152f7e54f prog_id: 59 target_name: task Signed-off-by:
-
- 20 Aug, 2020 10 commits
-
-
Andrii Nakryiko authored
Record which built-ins are optional and needed for some of recent BPF CO-RE subtests. Document Clang diff that fixed corner-case issue with __builtin_btf_type_id(). Suggested-by:
Daniel Borkmann <daniel@iogearbox.net> Acked-by:
-
Andrii Nakryiko authored
GCC 4.9 seems to be more strict in some regards. Fix two minor issue it reported. Fixes: 1c1052e0 ("tools/testing/selftests/bpf: Add self-tests for new helper bpf_get_ns_current_pid_tgid.") Fixes: 2d7824ff ("selftests: bpf: Add test for sk_assign") Signed-off-by:
-
Andrii Nakryiko authored
GCC compilers older than version 5 don't support __builtin_mul_overflow yet. Given GCC 4.9 is the minimal supported compiler for building kernel and the fact that libbpf is a dependency of resolve_btfids, which is dependency of CONFIG_DEBUG_INFO_BTF=y, this needs to be handled. This patch fixes the issue by falling back to slower detection of integer overflow in such cases. Fixes: 029258d7 ("libbpf: Remove any use of reallocarray() in libbpf") Signed-off-by:
-
Andrii Nakryiko authored
BPF_CALL | BPF_JMP32 is explicitly not allowed by verifier for BPF helper calls, so don't detect it as a valid call. Also drop the check on func_id pointer, as it's currently always non-null. Fixes: 109cea5a ("libbpf: Sanitize BPF program code for bpf_probe_read_{kernel, user}[_str]") Reported-by:
-
Daniel Borkmann authored
Alexei Starovoitov says: ==================== This patch set is the first real user of user mode driver facility. The general use case for user mode driver is to ship vmlinux with preloaded BPF programs. In this particular case the user mode driver populates bpffs instance with two BPF iterators. In several months BPF_LSM project would need to preload the kernel with its own set of BPF programs and attach to LSM hooks instead of bpffs. BPF iterators and BPF_LSM are unstable from uapi perspective. They are tracing based and peek into arbitrary kernel data structures. One can question why a kernel module cannot embed BPF programs inside. The reason is that libbpf is necessary to load them. First libbpf loads BPF Type Format, then creates BPF maps, populates them. Then it relocates code sections inside BPF programs, loads BPF programs, and finally attaches them to events. Theoretically libbpf can be rewritten to work in the kernel, but that is massive undertaking. The maintenance of in-kernel libbpf and user space libbpf would be another challenge. Another obstacle to embedding BPF programs into kernel module is sys_bpf api. Loading of programs, BTF, maps goes through the verifier. It validates and optimizes the code. It's possible to provide in-kernel api to all of sys_bpf commands (load progs, create maps, update maps, load BTF, etc), but that is huge amount of work and forever maintenance headache. Hence the decision is to ship vmlinux with user mode drivers that load BPF programs. Just like kernel modules extend vmlinux BPF programs are safe extensions of the kernel and some of them need to ship with vmlinux. This patch set adds a kernel module with user mode driver that populates bpffs with two BPF iterators. $ mount bpffs /my/bpffs/ -t bpf $ ls -la /my/bpffs/ total 4 drwxrwxrwt 2 root root 0 Jul 2 00:27 . drwxr-xr-x 19 root root 4096 Jul 2 00:09 .. -rw------- 1 root root 0 Jul 2 00:27 maps.debug -rw------- 1 root root 0 Jul 2 00:27 progs.debug The user mode driver will load BPF Type Formats, create BPF maps, populate BPF maps, load two BPF programs, attach them to BPF iterators, and finally send two bpf_link IDs back to the kernel. The kernel will pin two bpf_links into newly mounted bpffs instance under names "progs.debug" and "maps.debug". These two files become human readable. $ cat /my/bpffs/progs.debug id name attached 11 dump_bpf_map bpf_iter_bpf_map 12 dump_bpf_prog bpf_iter_bpf_prog 27 test_pkt_access 32 test_main test_pkt_access test_pkt_access 33 test_subprog1 test_pkt_access_subprog1 test_pkt_access 34 test_subprog2 test_pkt_access_subprog2 test_pkt_access 35 test_subprog3 test_pkt_access_subprog3 test_pkt_access 36 new_get_skb_len get_skb_len test_pkt_access 37 new_get_skb_ifindex get_skb_ifindex test_pkt_access 38 new_get_constant get_constant test_pkt_access The BPF program dump_bpf_prog() in iterators.bpf.c is printing this data about all BPF programs currently loaded in the system. This information is unstable and will change from kernel to kernel. In some sence this output is similar to 'bpftool prog show' that is using stable api to retreive information about BPF programs. The BPF subsytems grows quickly and there is always demand to show as much info about BPF things as possible. But we cannot expose all that info via stable uapi of bpf syscall, since the details change so much. Right now a BPF program can be attached to only one other BPF program. Folks are working on patches to enable multi-attach, but for debugging it's necessary to see the current state. There is no uapi for that, but above output shows it: 37 new_get_skb_ifindex get_skb_ifindex test_pkt_access 38 new_get_constant get_constant test_pkt_access [1] [2] [3] [1] is the full name of BPF prog from BTF. [2] is the name of function inside target BPF prog. [3] is the name of target BPF prog. [2] and [3] are not exposed via uapi, since they will change from single to multi soon. There are many other cases where bpf internals are useful for debugging, but shouldn't be exposed via uapi due to high rate of changes. systemd mounts /sys/fs/bpf at the start, so this kernel module with user mode driver needs to be available early. BPF_LSM most likely would need to preload BPF programs even earlier. Few interesting observations: - though bpffs comes with two human readble files "progs.debug" and "maps.debug" they can be removed. 'rm -f /sys/fs/bpf/progs.debug' will remove bpf_link and kernel will automatically unload corresponding BPF progs, maps, BTFs. In the future '-o remount' will be able to restore them. This is not implemented yet. - 'ps aux|grep bpf_preload' shows nothing. User mode driver loaded BPF iterators and exited. Nothing is lingering in user space at this point. - We can consider giving 0644 permissions to "progs.debug" and "maps.debug" to allow unprivileged users see BPF things loaded in the system. We cannot do so with "bpftool prog show", since it's using cap_sys_admin parts of bpf syscall. - The functionality split between core kernel, bpf_preload kernel module and user mode driver is very similar to bpfilter style of interaction. - Similar BPF iterators can be used as unstable extensions to /proc. Like mounting /proc can prepopolate some subdirectory in there with a BPF iterator that will print QUIC sockets instead of tcp and udp. Changelog: v5->v6: - refactored Makefiles with Andrii's help - switched to explicit $(MAKE) style - switched to userldlibs instead of userldflags - fixed build issue with libbpf Makefile due to invocation from kbuild - fixed menuconfig order as spotted by Daniel - introduced CONFIG_USERMODE_DRIVER bool that is selected by bpfilter and bpf_preload v4->v5: - addressed Song and Andrii feedback. s/pages/max_entries/ v3->v4: - took THIS_MODULE in patch 3 as suggested by Daniel to simplify the code. - converted BPF iterator to use BTF (when available) to print full BPF program name instead of 16-byte truncated version. This is something I've been using drgn scripts for. Take a look at get_name() in iterators.bpf.c to see how short it is comparing to what user space bpftool would have to do to print the same full name: . get prog info via obj_info_by_fd . do get_fd_by_id from info->btf_id . fetch potentially large BTF of the program from the kernel . parse that BTF in user space to figure out all type boundaries and string section . read info->func_info to get btf_id of func_proto from there . find that btf_id in the parsed BTF That's quite a bit work for bpftool comparing to few lines in get_name(). I guess would be good to make bpftool do this info extraction anyway. While doing this BTF reading in the kernel realized that the verifier is not smart enough to follow double pointers (added to my todo list), otherwise get_name() would have been even shorter. v2->v3: - fixed module unload race (Daniel) - added selftest (Daniel) - fixed build bot warning v1->v2: - changed names to 'progs.debug' and 'maps.debug' to hopefully better indicate instability of the text output. Having dot in the name also guarantees that these special files will not conflict with normal bpf objects pinned in bpffs, since dot is disallowed for normal pins. - instead of hard coding link_name in the core bpf moved into UMD. - cleanedup error handling. - addressed review comments from Yonghong and Andrii. ==================== Signed-off-by: -
Alexei Starovoitov authored
Add a test that mounts two bpffs instances and checks progs.debug and maps.debug for sanity data. Signed-off-by:
-
Alexei Starovoitov authored
Add kernel module with user mode driver that populates bpffs with BPF iterators. $ mount bpffs /my/bpffs/ -t bpf $ ls -la /my/bpffs/ total 4 drwxrwxrwt 2 root root 0 Jul 2 00:27 . drwxr-xr-x 19 root root 4096 Jul 2 00:09 .. -rw------- 1 root root 0 Jul 2 00:27 maps.debug -rw------- 1 root root 0 Jul 2 00:27 progs.debug The user mode driver will load BPF Type Formats, create BPF maps, populate BPF maps, load two BPF programs, attach them to BPF iterators, and finally send two bpf_link IDs back to the kernel. The kernel will pin two bpf_links into newly mounted bpffs instance under names "progs.debug" and "maps.debug". These two files become human readable. $ cat /my/bpffs/progs.debug id name attached 11 dump_bpf_map bpf_iter_bpf_map 12 dump_bpf_prog bpf_iter_bpf_prog 27 test_pkt_access 32 test_main test_pkt_access test_pkt_access 33 test_subprog1 test_pkt_access_subprog1 test_pkt_access 34 test_subprog2 test_pkt_access_subprog2 test_pkt_access 35 test_subprog3 test_pkt_access_subprog3 test_pkt_access 36 new_get_skb_len get_skb_len test_pkt_access 37 new_get_skb_ifindex get_skb_ifindex test_pkt_access 38 new_get_constant get_constant test_pkt_access The BPF program dump_bpf_prog() in iterators.bpf.c is printing this data about all BPF programs currently loaded in the system. This information is unstable and will change from kernel to kernel as ".debug" suffix conveys. Signed-off-by:
-
Alexei Starovoitov authored
The program and map iterators work similar to seq_file-s. Once the program is pinned in bpffs it can be read with "cat" tool to print human readable output. In this case about BPF programs and maps. For example: $ cat /sys/fs/bpf/progs.debug id name attached 5 dump_bpf_map bpf_iter_bpf_map 6 dump_bpf_prog bpf_iter_bpf_prog $ cat /sys/fs/bpf/maps.debug id name max_entries 3 iterator.rodata 1 To avoid kernel build dependency on clang 10 separate bpf skeleton generation into manual "make" step and instead check-in generated .skel.h into git. Unlike 'bpftool prog show' in-kernel BTF name is used (when available) to print full name of BPF program instead of 16-byte truncated name. Signed-off-by:
-
Alexei Starovoitov authored
Refactor the code a bit to extract bpf_link_by_id() helper. It's similar to existing bpf_prog_by_id(). Signed-off-by:
Song Liu <songliubraving@fb.com> Link: https://lore.kernel.org/bpf/20200819042759.51280-2-alexei.starovoitov@gmail.com
-
Xu Wang authored
Simplify the return expression. Signed-off-by:
Xu Wang <vulab@iscas.ac.cn> Signed-off-by:
-
- 19 Aug, 2020 11 commits
-
-
Alexei Starovoitov authored
Andrii Nakryiko says: ==================== This patch set adds libbpf support for two new classes of CO-RE relocations: type-based (TYPE_EXISTS/TYPE_SIZE/TYPE_ID_LOCAL/TYPE_ID_TARGET) and enum value-vased (ENUMVAL_EXISTS/ENUMVAL_VALUE): - TYPE_EXISTS allows to detect presence in kernel BTF of a locally-recorded BTF type. Useful for feature detection (new functionality often comes with new internal kernel types), as well as handling type renames and bigger refactorings. - TYPE_SIZE allows to get the real size (in bytes) of a specified kernel type. Useful for dumping internal structure as-is through perfbuf or ringbuf. - TYPE_ID_LOCAL/TYPE_ID_TARGET allow to capture BTF type ID of a BTF type in program's BTF or kernel BTF, respectively. These could be used for high-performance and space-efficient generic data dumping/logging by relying on small and cheap BTF type ID as a data layout descriptor, for post-processing on user-space side. - ENUMVAL_EXISTS can be used for detecting the presence of enumerator value in kernel's enum type. Most direct application is to detect BPF helper support in kernel. - ENUMVAL_VALUE allows to relocate real integer value of kernel enumerator value, which is subject to change (e.g., always a potential issue for internal, non-UAPI, kernel enums). I've indicated potential applications for these relocations, but relocations themselves are generic and unassuming and are designed to work correctly even in unintended applications. Furthermore, relocated values become constants, known to the verifier and could and would be used for dead branch code detection and elimination. This makes them ideal to do all sorts of feature detection and guarding functionality that's not available on some older (but still supported by BPF program) kernels, while having to compile and maintain one unified source code. Selftests are added for all the new features. Selftests utilizing new Clang built-ins are designed such that they will compile with older Clangs and will be skipped during test runs. So this shouldn't cause any build and test failures on systems with slightly outdated Clang compiler. LLVM patches adding these relocation in Clang: - __builtin_btf_type_id() ([0], [1], [2]); - __builtin_preserve_type_info(), __builtin_preserve_enum_value() ([3], [4]). [0] https://reviews.llvm.org/D74572 [1] https://reviews.llvm.org/D74668 [2] https://reviews.llvm.org/D85174 [3] https://reviews.llvm.org/D83878 [4] https://reviews.llvm.org/D83242 v2->v3: - fix feature detection for __builtin_btf_type_id() test (Yonghong); - fix extra empty lines at the end of files (Yonghong); v1->v2: - selftests detect built-in support and are skipped if not found (Alexei). ==================== Signed-off-by: -
Andrii Nakryiko authored
Add tests validating existence and value relocations for enum value-based relocations. If __builtin_preserve_enum_value() built-in is not supported, skip tests. Signed-off-by:
-
Andrii Nakryiko authored
Implement two relocations of a new enumerator value-based CO-RE relocation kind: ENUMVAL_EXISTS and ENUMVAL_VALUE. First, ENUMVAL_EXISTS, allows to detect the presence of a named enumerator value in the target (kernel) BTF. This is useful to do BPF helper/map/program type support detection from BPF program side. bpf_core_enum_value_exists() macro helper is provided to simplify built-in usage. Second, ENUMVAL_VALUE, allows to capture enumerator integer value and relocate it according to the target BTF, if it changes. This is useful to have a guarantee against intentional or accidental re-ordering/re-numbering of some of the internal (non-UAPI) enumerations, where kernel developers don't care about UAPI backwards compatiblity concerns. bpf_core_enum_value() allows to capture this succinctly and use correct enum values in code. LLVM uses ldimm64 instruction to capture enumerator value-based relocations, so add support for ldimm64 instruction patching as well. Signed-off-by:
-
Andrii Nakryiko authored
Add tests for BTF type ID relocations. To allow testing this, enhance core_relo.c test runner to allow dynamic initialization of test inputs. If Clang doesn't have necessary support for new functionality, test is skipped. Signed-off-by:
-
Andrii Nakryiko authored
Add selftests for TYPE_EXISTS and TYPE_SIZE relocations, testing correctness of relocations and handling of type compatiblity/incompatibility. If __builtin_preserve_type_info() is not supported by compiler, skip tests. Signed-off-by:
-
Andrii Nakryiko authored
Implement support for TYPE_EXISTS/TYPE_SIZE/TYPE_ID_LOCAL/TYPE_ID_REMOTE relocations. These are examples of type-based relocations, as opposed to field-based relocations supported already. The difference is that they are calculating relocation values based on the type itself, not a field within a struct/union. Type-based relos have slightly different semantics when matching local types to kernel target types, see comments in bpf_core_types_are_compat() for details. Their behavior on failure to find target type in kernel BTF also differs. Instead of "poisoning" relocatable instruction and failing load subsequently in kernel, they return 0 (which is rarely a valid return result, so user BPF code can use that to detect success/failure of the relocation and deal with it without extra "guarding" relocations). Also, it's always possible to check existence of the type in target kernel with TYPE_EXISTS relocation, similarly to a field-based FIELD_EXISTS. TYPE_ID_LOCAL relocation is a bit special in that it always succeeds (barring any libbpf/Clang bugs) and resolved to BTF ID using **local** BTF info of BPF program itself. Tests in subsequent patches demonstrate the usage and semantics of new relocations. Signed-off-by:
-
Maciej Żenczykowski authored
This reduces likelihood of incorrect use. Test: builds Signed-off-by:
Maciej Żenczykowski <maze@google.com> Signed-off-by:
-
Maciej Żenczykowski authored
This provides a minor performance boost by virtue of inlining instead of cross module function calls. Test: builds Signed-off-by:
-
Maciej Żenczykowski authored
This reduces likelihood of incorrect use. Test: builds Signed-off-by:
-
Alexei Starovoitov authored
Andrii Nakryiko says: ==================== Get rid of two feature detectors: reallocarray and libelf-mmap. Optional feature detections complicate libbpf Makefile and cause more troubles for various applications that want to integrate libbpf as part of their build. Patch #1 replaces all reallocarray() uses into libbpf-internal reallocarray() implementation. Patches #2 and #3 makes sure we won't re-introduce reallocarray() accidentally. Patch #2 also removes last use of libbpf_internal.h header inside bpftool. There is still nlattr.h that's used by both libbpf and bpftool, but that's left for a follow up patch to split. Patch #4 removed libelf-mmap feature detector and all its uses, as it's trivial to handle missing mmap support in libbpf, the way objtool has been doing it for a while. v1->v2 and v2->v3: - rebase to latest bpf-next (Alexei). ==================== Signed-off-by: -
Andrii Nakryiko authored
It's trivial to handle missing ELF_C_MMAP_READ support in libelf the way that objtool has solved it in ("774bec3f objtool: Add fallback from ELF_C_READ_MMAP to ELF_C_READ"). So instead of having an entire feature detector for that, just do what objtool does for perf and libbpf. And keep their Makefiles a bit simpler. Signed-off-by:
-
