- 29 Jun, 2022 8 commits
-
-
Stanislav Fomichev authored
lsm_cgroup/ is the prefix for BPF_LSM_CGROUP. Acked-by:
Martin KaFai Lau <kafai@fb.com> Acked-by:
Andrii Nakryiko <andrii@kernel.org> Signed-off-by:
Stanislav Fomichev <sdf@google.com> Link: https://lore.kernel.org/r/20220628174314.1216643-9-sdf@google.comSigned-off-by:
Alexei Starovoitov <ast@kernel.org>
-
Stanislav Fomichev authored
Has been slowly getting out of sync, let's update it. resolve_btfids usage has been updated to match the header changes. Also bring new parts of tools/include/uapi/linux/bpf.h. Acked-by:
-
Stanislav Fomichev authored
I don't see how to make it nice without introducing btf id lists for the hooks where these helpers are allowed. Some LSM hooks work on the locked sockets, some are triggering early and don't grab any locks, so have two lists for now: 1. LSM hooks which trigger under socket lock - minority of the hooks, but ideal case for us, we can expose existing BTF-based helpers 2. LSM hooks which trigger without socket lock, but they trigger early in the socket creation path where it should be safe to do setsockopt without any locks 3. The rest are prohibited. I'm thinking that this use-case might be a good gateway to sleeping lsm cgroup hooks in the future. We can either expose lock/unlock operations (and add tracking to the verifier) or have another set of bpf_setsockopt wrapper that grab the locks and might sleep. Reviewed-by:
-
Stanislav Fomichev authored
We have two options: 1. Treat all BPF_LSM_CGROUP the same, regardless of attach_btf_id 2. Treat BPF_LSM_CGROUP+attach_btf_id as a separate hook point I was doing (2) in the original patch, but switching to (1) here: * bpf_prog_query returns all attached BPF_LSM_CGROUP programs regardless of attach_btf_id * attach_btf_id is exported via bpf_prog_info Reviewed-by:
-
Stanislav Fomichev authored
Previous patch adds 1:1 mapping between all 211 LSM hooks and bpf_cgroup program array. Instead of reserving a slot per possible hook, reserve 10 slots per cgroup for lsm programs. Those slots are dynamically allocated on demand and reclaimed. struct cgroup_bpf { struct bpf_prog_array * effective[33]; /* 0 264 */ /* --- cacheline 4 boundary (256 bytes) was 8 bytes ago --- */ struct hlist_head progs[33]; /* 264 264 */ /* --- cacheline 8 boundary (512 bytes) was 16 bytes ago --- */ u8 flags[33]; /* 528 33 */ /* XXX 7 bytes hole, try to pack */ struct list_head storages; /* 568 16 */ /* --- cacheline 9 boundary (576 bytes) was 8 bytes ago --- */ struct bpf_prog_array * inactive; /* 584 8 */ struct percpu_ref refcnt; /* 592 16 */ struct work_struct release_work; /* 608 72 */ /* size: 680, cachelines: 11, members: 7 */ /* sum members: 673, holes: 1, sum holes: 7 */ /* last cacheline: 40 bytes */ }; Reviewed-by: -
Stanislav Fomichev authored
Allow attaching to lsm hooks in the cgroup context. Attaching to per-cgroup LSM works exactly like attaching to other per-cgroup hooks. New BPF_LSM_CGROUP is added to trigger new mode; the actual lsm hook we attach to is signaled via existing attach_btf_id. For the hooks that have 'struct socket' or 'struct sock' as its first argument, we use the cgroup associated with that socket. For the rest, we use 'current' cgroup (this is all on default hierarchy == v2 only). Note that for some hooks that work on 'struct sock' we still take the cgroup from 'current' because some of them work on the socket that hasn't been properly initialized yet. Behind the scenes, we allocate a shim program that is attached to the trampoline and runs cgroup effective BPF programs array. This shim has some rudimentary ref counting and can be shared between several programs attaching to the same lsm hook from different cgroups. Note that this patch bloats cgroup size because we add 211 cgroup_bpf_attach_type(s) for simplicity sake. This will be addressed in the subsequent patch. Also note that we only add non-sleepable flavor for now. To enable sleepable use-cases, bpf_prog_run_array_cg has to grab trace rcu, shim programs have to be freed via trace rcu, cgroup_bpf.effective should be also trace-rcu-managed + maybe some other changes that I'm not aware of. Reviewed-by:
-
Stanislav Fomichev authored
This lets us reclaim some space to be used by new cgroup lsm slots. Before: struct cgroup_bpf { struct bpf_prog_array * effective[23]; /* 0 184 */ /* --- cacheline 2 boundary (128 bytes) was 56 bytes ago --- */ struct list_head progs[23]; /* 184 368 */ /* --- cacheline 8 boundary (512 bytes) was 40 bytes ago --- */ u32 flags[23]; /* 552 92 */ /* XXX 4 bytes hole, try to pack */ /* --- cacheline 10 boundary (640 bytes) was 8 bytes ago --- */ struct list_head storages; /* 648 16 */ struct bpf_prog_array * inactive; /* 664 8 */ struct percpu_ref refcnt; /* 672 16 */ struct work_struct release_work; /* 688 32 */ /* size: 720, cachelines: 12, members: 7 */ /* sum members: 716, holes: 1, sum holes: 4 */ /* last cacheline: 16 bytes */ }; After: struct cgroup_bpf { struct bpf_prog_array * effective[23]; /* 0 184 */ /* --- cacheline 2 boundary (128 bytes) was 56 bytes ago --- */ struct hlist_head progs[23]; /* 184 184 */ /* --- cacheline 5 boundary (320 bytes) was 48 bytes ago --- */ u8 flags[23]; /* 368 23 */ /* XXX 1 byte hole, try to pack */ /* --- cacheline 6 boundary (384 bytes) was 8 bytes ago --- */ struct list_head storages; /* 392 16 */ struct bpf_prog_array * inactive; /* 408 8 */ struct percpu_ref refcnt; /* 416 16 */ struct work_struct release_work; /* 432 72 */ /* size: 504, cachelines: 8, members: 7 */ /* sum members: 503, holes: 1, sum holes: 1 */ /* last cacheline: 56 bytes */ }; Suggested-by:Jakub Sitnicki <jakub@cloudflare.com> Reviewed-by:
-
Stanislav Fomichev authored
I'll be adding lsm cgroup specific helpers that grab trampoline mutex. No functional changes. Reviewed-by:
-
- 28 Jun, 2022 17 commits
-
-
Alexei Starovoitov authored
Andrii Nakryiko says: ==================== This patch set removes all the deprecated APIs in preparation for 1.0 release. It also makes libbpf_set_strict_mode() a no-op (but keeps it to let per-1.0 applications buildable and dynamically linkable against libbpf 1.0 if they were already libbpf-1.0 ready) and starts enforcing all the behaviors that were previously opt-in through libbpf_set_strict_mode(). xsk.{c,h} parts that are now properly provided by libxdp ([0]) are still used by xdpxceiver.c in selftest/bpf, so I've moved xsk.{c,h} with barely any changes to under selftests/bpf. Other than that, apart from removing all the LIBBPF_DEPRECATED-marked APIs, there is a bunch of internal clean ups allowed by that. I've also "restored" libbpf.map inheritance chain while removing all the deprecated APIs. I think that's the right way to do this, as applications using libbpf as shared library but not relying on any deprecated APIs (i.e., "good citizens" that prepared for libbpf 1.0 release ahead of time to minimize disruption) should be able to link both against 0.x and 1.x versions. Either way, it doesn't seem to break anything and preserve a history on when each "surviving" API was added. [0] https://github.com/xdp-project/xdp-tools/tree/master/lib/libxdp v1->v2: - rebase on latest bpf-next now that Jiri's perf patches landed; - fix xsk.o dependency in Makefile to ensure libbpf headers are installed reliably. ==================== Signed-off-by: -
Andrii Nakryiko authored
Seems like we missed to add 2 APIs to libbpf.map and another API was misspelled. Fix it in libbpf.map. Signed-off-by:
-
Andrii Nakryiko authored
Remove support for legacy features and behaviors that previously had to be disabled by calling libbpf_set_strict_mode(): - legacy BPF map definitions are not supported now; - RLIMIT_MEMLOCK auto-setting, if necessary, is always on (but see libbpf_set_memlock_rlim()); - program name is used for program pinning (instead of section name); - cleaned up error returning logic; - entry BPF programs should have SEC() always. Signed-off-by: -
Andrii Nakryiko authored
Libbpf 1.0 stops support legacy-style BPF map definitions. Selftests has been migrated away from using legacy BPF map definitions except for two selftests, to make sure that legacy functionality still worked in pre-1.0 libbpf. Now it's time to let those tests go as libbpf 1.0 is imminent. Signed-off-by:
-
Andrii Nakryiko authored
Get rid of sloppy prefix logic and remove deprecated xdp_{devmap,cpumap} sections. Signed-off-by: -
Andrii Nakryiko authored
Clean up internals that had to deal with the possibility of multi-instance bpf_programs. Libbpf 1.0 doesn't support this, so all this is not necessary now and can be simplified. Signed-off-by:
-
Andrii Nakryiko authored
Keep the LIBBPF_DEPRECATED_SINCE macro "framework" for future deprecations, but clean up 0.x related helper macros. Signed-off-by:
-
Andrii Nakryiko authored
Remove all the public APIs that are related to creating multi-instance bpf_programs through custom preprocessing callback and generally working with them. Also remove all the bpf_{object,map,program}__[set_]priv() APIs. Signed-off-by: -
Andrii Nakryiko authored
Remove a bunch of high-level bpf_object/bpf_map/bpf_program related APIs. All the APIs related to private per-object/map/prog state, program preprocessing callback, and generally everything multi-instance related is removed in a separate patch. Signed-off-by:
-
Andrii Nakryiko authored
Remove prog_info_linear-related APIs previously used by perf. Signed-off-by:
-
Andrii Nakryiko authored
Remove deprecated perfbuf APIs and clean up opts structs. Signed-off-by:
-
Andrii Nakryiko authored
Get rid of deprecated BTF-related APIs. Signed-off-by:
-
Andrii Nakryiko authored
Get rid of deprecated feature-probing APIs. Signed-off-by:
-
Andrii Nakryiko authored
Get rid of deprecated bpf_set_link*() and bpf_get_link*() APIs. Signed-off-by:
-
Andrii Nakryiko authored
Drop low-level APIs as well as high-level (and very confusingly named) BPF object loading bpf_prog_load_xattr() and bpf_prog_load_deprecated() APIs. Signed-off-by:
-
Andrii Nakryiko authored
Remove deprecated xsk APIs from libbpf. But given we have selftests relying on this, move those files (with minimal adjustments to make them compilable) under selftests/bpf. We also remove all the removed APIs from libbpf.map, while overall keeping version inheritance chain, as most APIs are backwards compatible so there is no need to reassign them as LIBBPF_1.0.0 versions. Cc: Magnus Karlsson <magnus.karlsson@gmail.com> Signed-off-by:
-
John Fastabend authored
syzbot reproduced the bug ... BUG: sleeping function called from invalid context at kernel/workqueue.c:3010 ... with the following stack trace fragment ... start_flush_work kernel/workqueue.c:3010 [inline] __flush_work+0x109/0xb10 kernel/workqueue.c:3074 __cancel_work_timer+0x3f9/0x570 kernel/workqueue.c:3162 sk_psock_stop+0x4cb/0x630 net/core/skmsg.c:802 sock_map_destroy+0x333/0x760 net/core/sock_map.c:1581 inet_csk_destroy_sock+0x196/0x440 net/ipv4/inet_connection_sock.c:1130 __tcp_close+0xd5b/0x12b0 net/ipv4/tcp.c:2897 tcp_close+0x29/0xc0 net/ipv4/tcp.c:2909 ... introduced by d8616ee2. Do a quick trace of the code path and the bug is obvious: inet_csk_destroy_sock(sk) sk_prot->destroy(sk); <--- sock_map_destroy sk_psock_stop(, true); <--- true so cancel workqueue cancel_work_sync() <--- splat, because *_bh_disable() We can not call cancel_work_sync() from inside destroy path. So mark the sk_psock_stop call to skip this cancel_work_sync(). This will avoid the BUG, but means we may run sk_psock_backlog after or during the destroy op. We zapped the ingress_skb queue in sk_psock_stop (safe to do with local_bh_disable) so its empty and the sk_psock_backlog work item will not find any pkts to process here. However, because we are not going to wait for it or clear its ->state its possible it kicks off or is already running. This should be 'safe' up until psock drops its refcnt to psock->sk. The sock_put() that drops this reference is only done at psock destroy time from sk_psock_destroy(). This is done through workqueue when sk_psock_drop() is called on psock refnt reaches 0. And importantly sk_psock_destroy() does a cancel_work_sync(). So trivial fix works. I've had hit or miss luck reproducing this caught it once or twice with the provided reproducer when running with many runners. However, syzkaller is very good at reproducing so relying on syzkaller to verify fix. Fixes: d8616ee2 ("bpf, sockmap: Fix sk->sk_forward_alloc warn_on in sk_stream_kill_queues") Reported-by: syzbot+140186ceba0c496183bc@syzkaller.appspotmail.com Suggested-by:
Hillf Danton <hdanton@sina.com> Signed-off-by:
John Fastabend <john.fastabend@gmail.com> Signed-off-by:
Daniel Borkmann <daniel@iogearbox.net> Cc: Wang Yufen <wangyufen@huawei.com> Link: https://lore.kernel.org/bpf/20220628035803.317876-1-john.fastabend@gmail.com
-
- 24 Jun, 2022 7 commits
-
-
Daniel Müller authored
BPF type compatibility checks (bpf_core_types_are_compat()) are currently duplicated between kernel and user space. That's a historical artifact more than intentional doing and can lead to subtle bugs where one implementation is adjusted but another is forgotten. That happened with the enum64 work, for example, where the libbpf side was changed (commit 23b2a3a8 ("libbpf: Add enum64 relocation support")) to use the btf_kind_core_compat() helper function but the kernel side was not (commit 6089fb32 ("bpf: Add btf enum64 support")). This patch addresses both the duplication issue, by merging both implementations and moving them into relo_core.c, and fixes the alluded to kind check (by giving preference to libbpf's already adjusted logic). For discussion of the topic, please refer to: https://lore.kernel.org/bpf/CAADnVQKbWR7oarBdewgOBZUPzryhRYvEbkhyPJQHHuxq=0K1gw@mail.gmail.com/T/#mcc99f4a33ad9a322afaf1b9276fb1f0b7add9665 Changelog: v1 -> v2: - limited libbpf recursion limit to 32 - changed name to __bpf_core_types_are_compat - included warning previously present in libbpf version - merged kernel and user space changes into a single patch Signed-off-by:
Daniel Müller <deso@posteo.net> Signed-off-by:
-
Shahab Vahedi authored
A minor typo fix to include "| BPF_LD" into its previous code phrase: ``BPF_IND`` | BPF_LD --> ``BPF_IND | BPF_LD`` Signed-off-by:
Shahab Vahedi <shahab@synopsys.com> Signed-off-by:
-
Andrii Nakryiko authored
Jiri Olsa says: ==================== hi, sending change we discussed some time ago [1] to get rid of some deprecated functions we use in perf prologue code. Despite the gloomy discussion I think the final code does not look that bad ;-) This patchset removes following libbpf functions from perf: bpf_program__set_prep bpf_program__nth_fd struct bpf_prog_prep_result v5 changes: - squashed patches together so we don't break bisection [Arnaldo] v4 changes: - fix typo [Andrii] v3 changes: - removed R0/R1 zero init in libbpf_prog_prepare_load_fn, because it's not needed [Andrii] - rebased/post on top of bpf-next/master which now has all the needed perf/core changes v2 changes: - use fallback section prog handler, so we don't need to use section prefix [Andrii] - realloc prog->insns array in bpf_program__set_insns [Andrii] - squash patch 1 from previous version with bpf_program__set_insns change [Daniel] - patch 3 already merged [Arnaldo] - added more comments thanks, jirka [1] https://lore.kernel.org/bpf/CAEf4BzaiBO3_617kkXZdYJ8hS8YF--ZLgapNbgeeEJ-pY0H88g@mail.gmail.com/ ==================== Signed-off-by: -
Jiri Olsa authored
Some functions we use for bpf prologue generation are going to be deprecated. This change reworks current code not to use them. We need to replace following functions/struct: bpf_program__set_prep bpf_program__nth_fd struct bpf_prog_prep_result Currently we use bpf_program__set_prep to hook perf callback before program is loaded and provide new instructions with the prologue. We replace this function/ality by taking instructions for specific program, attaching prologue to them and load such new ebpf programs with prologue using separate bpf_prog_load calls (outside libbpf load machinery). Before we can take and use program instructions, we need libbpf to actually load it. This way we get the final shape of its instructions with all relocations and verifier adjustments). There's one glitch though.. perf kprobe program already assumes generated prologue code with proper values in argument registers, so loading such program directly will fail in the verifier. That's where the fallback pre-load handler fits in and prepends the initialization code to the program. Once such program is loaded we take its instructions, cut off the initialization code and prepend the prologue. I know.. sorry ;-) To have access to the program when loading this patch adds support to register 'fallback' section handler to take care of perf kprobe programs. The fallback means that it handles any section definition besides the ones that libbpf handles. The handler serves two purposes: - allows perf programs to have special arguments in section name - allows perf to use pre-load callback where we can attach init code (zeroing all argument registers) to each perf program Suggested-by:Jiri Olsa <jolsa@kernel.org> Signed-off-by:
Arnaldo Carvalho de Melo <acme@redhat.com> Acked-by:
-
Eduard Zingerman authored
This test verifies that bpf_loop() inlining works as expected when address of `env->prog` is updated. This address is updated upon BPF program reallocation. Reallocation is handled by bpf_prog_realloc(), which reuses old memory if page boundary is not crossed. The value of `len` in the test is chosen to cross this boundary on bpf_loop() patching. Verify that the use-after-free bug in inline_bpf_loop() reported by Dan Carpenter is fixed. Signed-off-by:
Eduard Zingerman <eddyz87@gmail.com> Signed-off-by:
-
Eduard Zingerman authored
As reported by Dan Carpenter, the following statements in inline_bpf_loop() might cause a use-after-free bug: struct bpf_prog *new_prog; // ... new_prog = bpf_patch_insn_data(env, position, insn_buf, *cnt); // ... env->prog->insnsi[call_insn_offset].imm = callback_offset; The bpf_patch_insn_data() might free the memory used by env->prog. Fixes: 1ade2371 ("bpf: Inline calls to bpf_loop when callback is known") Reported-by:
Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by:
-
Simon Wang authored
Enhance readability a bit. Signed-off-by:
Simon Wang <wangchuanguo@inspur.com> Signed-off-by:
-
- 23 Jun, 2022 8 commits
-
-
Jörn-Thorben Hinz authored
test_sock_fields__detach() got called with a null pointer here when one of the CHECKs or ASSERTs up to the test_sock_fields__open_and_load() call resulted in a jump to the "done" label. A skeletons *__detach() is not safe to call with a null pointer, though. This led to a segfault. Go the easy route and only call test_sock_fields__destroy() which is null-pointer safe and includes detaching. Came across this while looking[1] to introduce the usage of bpf_tcp_helpers.h (included in progs/test_sock_fields.c) together with vmlinux.h. [1] https://lore.kernel.org/bpf/629bc069dd807d7ac646f836e9dca28bbc1108e2.camel@mailbox.tu-berlin.de/ Fixes: 8f50f16f ("selftests/bpf: Extend verifier and bpf_sock tests for dst_port loads") Signed-off-by:
Jörn-Thorben Hinz <jthinz@mailbox.tu-berlin.de> Signed-off-by:
-
Alexei Starovoitov authored
Jörn-Thorben Hinz says: ==================== This series corrects some inconveniences for a BPF TCP CC that implements and uses tcp_congestion_ops.cong_control(). Until now, such a CC did not have all necessary write access to struct sock and unnecessarily needed to implement cong_avoid(). v4: - Remove braces around single statements after if - Don’t check pointer passed to bpf_link__destroy() v3: - Add a selftest writing sk_pacing_* - Add a selftest with incomplete tcp_congestion_ops - Add a selftest with unsupported get_info() - Remove an unused variable - Reword a comment about reg() in bpf_struct_ops_map_update_elem() v2: - Drop redundant check for required functions and just rely on tcp_register_congestion_control() (Martin KaFai Lau) ==================== Reviewed-by:
-
Jörn-Thorben Hinz authored
Test whether a TCP CC implemented in BPF providing get_info() is rejected correctly. get_info() is unsupported in a BPF CC. The check for required functions in a BPF CC has moved, this test ensures unsupported functions are still rejected correctly. Signed-off-by:
-
Jörn-Thorben Hinz authored
Test whether a TCP CC implemented in BPF providing neither cong_avoid() nor cong_control() is correctly rejected. This check solely depends on tcp_register_congestion_control() now, which is invoked during bpf_map__attach_struct_ops(). Signed-off-by:
-
Jörn-Thorben Hinz authored
Test whether a TCP CC implemented in BPF is allowed to write sk_pacing_rate and sk_pacing_status in struct sock. This is needed when cong_control() is implemented and used. Signed-off-by:
-
Jörn-Thorben Hinz authored
Remove the check for required and optional functions in a struct tcp_congestion_ops from bpf_tcp_ca.c. Rely on tcp_register_congestion_control() to reject a BPF CC that does not implement all required functions, as it will do for a non-BPF CC. When a CC implements tcp_congestion_ops.cong_control(), the alternate cong_avoid() is not in use in the TCP stack. Previously, a BPF CC was still forced to implement cong_avoid() as a no-op since it was non-optional in bpf_tcp_ca.c. Signed-off-by:
-
Jörn-Thorben Hinz authored
A CC that implements tcp_congestion_ops.cong_control() should be able to control sk_pacing_rate and set sk_pacing_status, since tcp_update_pacing_rate() is never called in this case. A built-in CC or one from a kernel module is already able to write to both struct sock members. For a BPF program, write access has not been allowed, yet. Signed-off-by:
-
Jian Shen authored
The prototype of .features is netdev_features_t, it should use NETIF_F_LLTX and NETIF_F_HW_VLAN_STAG_TX, not NETIF_F_LLTX_BIT and NETIF_F_HW_VLAN_STAG_TX_BIT. Fixes: cf204a71 ("bpf, testing: Introduce 'gso_linear_no_head_frag' skb_segment test") Signed-off-by:
Jian Shen <shenjian15@huawei.com> Acked-by:
-
