- 25 Aug, 2023 1 commit
-
-
Dave Marchevsky authored
It's straightforward to prove that kptr_struct_meta must be non-NULL for any valid call to these kfuncs: * btf_parse_struct_metas in btf.c creates a btf_struct_meta for any struct in user BTF with a special field (e.g. bpf_refcount, {rb,list}_node). These are stored in that BTF's struct_meta_tab. * __process_kf_arg_ptr_to_graph_node in verifier.c ensures that nodes have {rb,list}_node field and that it's at the correct offset. Similarly, check_kfunc_args ensures bpf_refcount field existence for node param to bpf_refcount_acquire. * So a btf_struct_meta must have been created for the struct type of node param to these kfuncs * That BTF and its struct_meta_tab are guaranteed to still be around. Any arbitrary {rb,list} node the BPF program interacts with either: came from bpf_obj_new or a collection removal kfunc in the same program, in which case the BTF is associated with the program and still around; or came from bpf_kptr_xchg, in which case the BTF was associated with the map and is still around Instead of silently continuing with NULL struct_meta, which caused confusing bugs such as those addressed by commit 2140a6e3 ("bpf: Set kptr_struct_meta for node param to list and rbtree insert funcs"), let's error out. Then, at runtime, we can confidently say that the implementations of these kfuncs were given a non-NULL kptr_struct_meta, meaning that special-field-specific functionality like bpf_obj_free_fields and the bpf_obj_drop change introduced later in this series are guaranteed to execute. This patch doesn't change functionality, just makes it easier to reason about existing functionality. Signed-off-by:Dave Marchevsky <davemarchevsky@fb.com> Acked-by:
Yonghong Song <yonghong.song@linux.dev> Link: https://lore.kernel.org/r/20230821193311.3290257-2-davemarchevsky@fb.comSigned-off-by:
Alexei Starovoitov <ast@kernel.org>
-
- 24 Aug, 2023 21 commits
-
-
Alexei Starovoitov authored
Pu Lehui says: ==================== Add support cpu v4 insns for RV64 Add support cpu v4 instructions for RV64. The relevant tests have passed as show bellow: Summary: 6/166 PASSED, 0 SKIPPED, 0 FAILED NOTE: ldsx_insn testcase uses fentry and needs to rely on ftrace direct call [0]. [0] https://lore.kernel.org/all/20230627111612.761164-1-suagrfillet@gmail.com/ v2: - Use temporary reg to avoid clobbering the source reg in movs_8/16 insns. (Björn) - Add Acked-by v1: https://lore.kernel.org/bpf/20230823231059.3363698-1-pulehui@huaweicloud.com ==================== Tested-by:
Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/r/20230824095001.3408573-1-pulehui@huaweicloud.comSigned-off-by:
-
Pu Lehui authored
Enable cpu v4 tests for RV64, and the relevant tests have passed. Signed-off-by:
Pu Lehui <pulehui@huawei.com> Acked-by:
Björn Töpel <bjorn@kernel.org> Link: https://lore.kernel.org/r/20230824095001.3408573-8-pulehui@huaweicloud.comSigned-off-by:
-
Pu Lehui authored
Add support unconditional bswap instruction. Since riscv is always little-endian, just treat the unconditional scenario the same as big-endian conversion. Signed-off-by:
-
Pu Lehui authored
Add support signed div/mod instructions for RV64. Signed-off-by:
-
Pu Lehui authored
Add support 32-bit offset jmp instruction for RV64. Signed-off-by:
-
Pu Lehui authored
Add support sign-extension mov instructions for RV64. Signed-off-by:
-
Pu Lehui authored
Add Support sign-extension load instructions for RV64. Signed-off-by:
-
Pu Lehui authored
For LDX_B/H/W, when zext has been inserted by verifier, it'll return 1, and no exception handling will continue. Also, when the offset is 12-bit value, the redundant zext inserted by the verifier is not removed. Fix both scenarios by moving down the removal of redundant zext. Signed-off-by:
-
Alexei Starovoitov authored
Toke Høiland-Jørgensen says: ==================== samples/bpf: Remove unmaintained XDP sample utilities The samples/bpf directory in the kernel tree started out as a way of showcasing different aspects of BPF functionality by writing small utility programs for each feature. However, as the BPF subsystem has matured, the preferred way of including userspace code with a feature has become the BPF selftests, which also have the benefit of being consistently run as part of the BPF CI system. As a result of this shift, the utilities in samples/bpf have seen little love, and have slowly bitrotted. There have been sporadic cleanup patches over the years, but it's clear that the utilities are far from maintained. For XDP in particular, some of the utilities have been used as benchmarking aids when implementing new kernel features, which seems to be the main reason they have stuck around; any updates the utilities have seen have been targeted at this use case. However, as the BPF subsystem as a whole has moved on, it has become increasingly difficult to incorporate new features into these utilities because they predate most of the modern BPF features (such as kfuncs and BTF). Rather than try to update these utilities and keep maintaining them in the kernel tree, we have ported the useful features of the utilities to the xdp-tools package. In the porting process we also updated the utilities to take advantage of modern BPF features, integrated them with libxdp, and polished the user interface. As these utilities are standalone tools, maintaining them out of tree is simpler, and we plan to keep maintaining them in the xdp-tools repo. To direct users of these utilities to the right place, this series removes the utilities from samples/bpf, leaving in place only a couple of utilities whose functionality have not yet been ported to xdp-tools. The xdp-tools repository is located on Github at the following URL: https://github.com/xdp-project/xdp-tools The commits in the series removes one utility each, explaining how the equivalent functionality can be obtained with xdp-tools. v2: - Add equivalent xdp-tools commands for each removed utility v3: - Add link to xdp-tools in the README Toke Høiland-Jørgensen (7): samples/bpf: Remove the xdp_monitor utility samples/bpf: Remove the xdp_redirect* utilities samples/bpf: Remove the xdp_rxq_info utility samples/bpf: Remove the xdp1 and xdp2 utilities samples/bpf: Remove the xdp_sample_pkts utility samples/bpf: Cleanup .gitignore samples/bpf: Add note to README about the XDP utilities moved to xdp-tools samples/bpf/.gitignore | 12 - samples/bpf/Makefile | 48 +- samples/bpf/README.rst | 6 + samples/bpf/xdp1_kern.c | 100 ---- samples/bpf/xdp1_user.c | 166 ------ samples/bpf/xdp2_kern.c | 125 ----- samples/bpf/xdp_monitor.bpf.c | 8 - samples/bpf/xdp_monitor_user.c | 118 ----- samples/bpf/xdp_redirect.bpf.c | 49 -- samples/bpf/xdp_redirect_cpu.bpf.c | 539 ------------------- samples/bpf/xdp_redirect_cpu_user.c | 559 -------------------- samples/bpf/xdp_redirect_map.bpf.c | 97 ---- samples/bpf/xdp_redirect_map_multi.bpf.c | 77 --- samples/bpf/xdp_redirect_map_multi_user.c | 232 -------- samples/bpf/xdp_redirect_map_user.c | 228 -------- samples/bpf/xdp_redirect_user.c | 172 ------ samples/bpf/xdp_rxq_info_kern.c | 140 ----- samples/bpf/xdp_rxq_info_user.c | 614 ---------------------- samples/bpf/xdp_sample_pkts_kern.c | 57 -- samples/bpf/xdp_sample_pkts_user.c | 196 ------- 20 files changed, 7 insertions(+), 3536 deletions(-) delete mode 100644 samples/bpf/xdp1_kern.c delete mode 100644 samples/bpf/xdp1_user.c delete mode 100644 samples/bpf/xdp2_kern.c delete mode 100644 samples/bpf/xdp_monitor.bpf.c delete mode 100644 samples/bpf/xdp_monitor_user.c delete mode 100644 samples/bpf/xdp_redirect.bpf.c delete mode 100644 samples/bpf/xdp_redirect_cpu.bpf.c delete mode 100644 samples/bpf/xdp_redirect_cpu_user.c delete mode 100644 samples/bpf/xdp_redirect_map.bpf.c delete mode 100644 samples/bpf/xdp_redirect_map_multi.bpf.c delete mode 100644 samples/bpf/xdp_redirect_map_multi_user.c delete mode 100644 samples/bpf/xdp_redirect_map_user.c delete mode 100644 samples/bpf/xdp_redirect_user.c delete mode 100644 samples/bpf/xdp_rxq_info_kern.c delete mode 100644 samples/bpf/xdp_rxq_info_user.c delete mode 100644 samples/bpf/xdp_sample_pkts_kern.c delete mode 100644 samples/bpf/xdp_sample_pkts_user.c ==================== Link: https://lore.kernel.org/r/20230824102255.1561885-1-toke@redhat.comSigned-off-by:
-
Toke Høiland-Jørgensen authored
To help users find the XDP utilities, add a note to the README about the new location and the conversion documentation in the commit messages. Signed-off-by:
Toke Høiland-Jørgensen <toke@redhat.com> Link: https://lore.kernel.org/r/20230824102255.1561885-8-toke@redhat.comSigned-off-by:
-
Toke Høiland-Jørgensen authored
Remove no longer present XDP utilities from .gitignore. Apart from the recently removed XDP utilities this also includes the previously removed xdpsock and xsk utilities. Signed-off-by:
-
Toke Høiland-Jørgensen authored
The functionality of this utility is covered by the xdpdump utility in xdp-tools. There's a slight difference in usage as the xdpdump utility's main focus is to dump packets before or after they are processed by an existing XDP program. However, xdpdump also has the --load-xdp-program switch, which will make it attach its own program if no existing program is loaded. With this, xdp_sample_pkts usage can be converted as: xdp_sample_pkts eth0 --> xdpdump --load-xdp-program eth0 To get roughly equivalent behaviour. Signed-off-by:
-
Toke Høiland-Jørgensen authored
The functionality of these utilities have been incorporated into the xdp-bench utility in xdp-tools. Equivalent functionality is: xdp1 eth0 --> xdp-bench drop -p parse-ip -l load-bytes eth0 xdp2 eth0 --> xdp-bench drop -p swap-macs eth0 Note that there's a slight difference in behaviour of those examples: the swap-macs operation of xdp-bench doesn't use the bpf_xdp_load_bytes() helper to load the packet data, whereas the xdp2 utility did so unconditionally. For the parse-ip action the use of bpf_xdp_load_bytes() can be selected by the '-l load-bytes' switch, with the difference that the xdp-bench utility will perform two separate calls to the helper, one to load the ethernet header and another to load the IP header; where the xdp1 utility only performed one call always loading 60 bytes of data. Signed-off-by:
-
Toke Høiland-Jørgensen authored
The functionality of this utility has been incorporated into the xdp-bench utility in xdp-tools, by way of the --rxq-stats argument to the 'drop', 'pass' and 'tx' commands of xdp-bench. Some examples of how to convert xdp_rxq_info invocations into equivalent xdp-bench commands: xdp_rxq_info -d eth0 --> xdp-bench pass --rxq-stats eth0 xdp_rxq_info -d eth0 -a XDP_DROP -m --> xdp-bench drop --rxq-stats -p swap-macs eth0 Signed-off-by:
-
Toke Høiland-Jørgensen authored
These utilities have all been ported to xdp-tools as functions of the xdp-bench utility. The four different utilities in samples are incorporated as separate subcommands to xdp-bench, with most of the command line parameters left intact, except that mandatory arguments are always positional in xdp-bench. For full usage details see the --help output of each command, or the xdp-bench man page. Some examples of how to convert usage to xdp-bench are: xdp_redirect eth0 eth1 --> xdp-bench redirect eth0 eth1 xdp_redirect_map eth0 eth1 --> xdp-bench redirect-map eth0 eth1 xdp_redirect_map_multi eth0 eth1 eth2 eth3 --> xdp-bench redirect-multi eth0 eth1 eth2 eth3 xdp_redirect_cpu -d eth0 -c 0 -c 1 --> xdp-bench redirect-cpu -c 0 -c 1 eth0 xdp_redirect_cpu -d eth0 -c 0 -c 1 -r eth1 --> xdp-bench redirect-cpu -c 0 -c 1 eth0 -r redirect -D eth1 Signed-off-by:
-
Toke Høiland-Jørgensen authored
This utility has been ported as-is to xdp-tools as 'xdp-monitor'. The only difference in usage between the samples and xdp-tools versions is that the '-v' command line parameter has been changed to '-e' in the xdp-tools version for consistency with the other utilities. Signed-off-by:
-
Yonghong Song authored
Add a local kptr test with no special fields in the struct. Without the previous patch, the following warning will hit: [ 44.683877] WARNING: CPU: 3 PID: 485 at kernel/bpf/syscall.c:660 bpf_obj_free_fields+0x220/0x240 [ 44.684640] Modules linked in: bpf_testmod(OE) [ 44.685044] CPU: 3 PID: 485 Comm: kworker/u8:5 Tainted: G OE 6.5.0-rc5-01703-g260d855e9b90 #248 [ 44.685827] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 [ 44.686693] Workqueue: events_unbound bpf_map_free_deferred [ 44.687297] RIP: 0010:bpf_obj_free_fields+0x220/0x240 [ 44.687775] Code: e8 55 17 1f 00 49 8b 74 24 08 4c 89 ef e8 e8 14 05 00 e8 a3 da e2 ff e9 55 fe ff ff 0f 0b e9 4e fe ff ff 0f 0b e9 47 fe ff ff <0f> 0b e8 d9 d9 e2 ff 31 f6 eb d5 48 83 c4 10 5b 41 5c e [ 44.689353] RSP: 0018:ffff888106467cb8 EFLAGS: 00010246 [ 44.689806] RAX: 0000000000000000 RBX: ffff888112b3a200 RCX: 0000000000000001 [ 44.690433] RDX: 0000000000000000 RSI: dffffc0000000000 RDI: ffff8881128ad988 [ 44.691094] RBP: 0000000000000002 R08: ffffffff81370bd0 R09: 1ffff110216231a5 [ 44.691643] R10: dffffc0000000000 R11: ffffed10216231a6 R12: ffff88810d68a488 [ 44.692245] R13: ffff88810767c288 R14: ffff88810d68a400 R15: ffff88810d68a418 [ 44.692829] FS: 0000000000000000(0000) GS:ffff8881f7580000(0000) knlGS:0000000000000000 [ 44.693484] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 44.693964] CR2: 000055c7f2afce28 CR3: 000000010fee4002 CR4: 0000000000370ee0 [ 44.694513] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 44.695102] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 44.695747] Call Trace: [ 44.696001] <TASK> [ 44.696183] ? __warn+0xfe/0x270 [ 44.696447] ? bpf_obj_free_fields+0x220/0x240 [ 44.696817] ? report_bug+0x220/0x2d0 [ 44.697180] ? handle_bug+0x3d/0x70 [ 44.697507] ? exc_invalid_op+0x1a/0x50 [ 44.697887] ? asm_exc_invalid_op+0x1a/0x20 [ 44.698282] ? btf_find_struct_meta+0xd0/0xd0 [ 44.698634] ? bpf_obj_free_fields+0x220/0x240 [ 44.699027] ? bpf_obj_free_fields+0x1e2/0x240 [ 44.699414] array_map_free+0x1a3/0x260 [ 44.699763] bpf_map_free_deferred+0x7b/0xe0 [ 44.700154] process_one_work+0x46d/0x750 [ 44.700523] worker_thread+0x49e/0x900 [ 44.700892] ? pr_cont_work+0x270/0x270 [ 44.701224] kthread+0x1ae/0x1d0 [ 44.701516] ? kthread_blkcg+0x50/0x50 [ 44.701860] ret_from_fork+0x34/0x50 [ 44.702178] ? kthread_blkcg+0x50/0x50 [ 44.702508] ret_from_fork_asm+0x11/0x20 [ 44.702880] </TASK> With the previous patch, there is no warnings. Signed-off-by: -
Yonghong Song authored
Currently, in function bpf_obj_free_fields(), for local kptr, a warning will be issued if the struct does not contain any special fields. But actually the kernel seems totally okay with a local kptr without any special fields. Permitting no special fields also aligns with future percpu kptr which also allows no special fields. Acked-by:
-
Andrii Nakryiko authored
Extracting btf_int_encoding() is only meaningful for BTF_KIND_INT, so we need to check that first before inferring signedness. Closes: https://github.com/libbpf/libbpf/issues/704Reported-by:
Lorenz Bauer <lmb@isovalent.com> Signed-off-by:
Andrii Nakryiko <andrii@kernel.org> Acked-by:
Martin KaFai Lau <martin.lau@kernel.org>
-
Andrii Nakryiko authored
It seems like it was forgotten to add uprobe_multi binary to .gitignore. Fix this trivial omission. Signed-off-by:
-
Daniel Xu authored
For bpf_object__pin_programs() there is bpf_object__unpin_programs(). Likewise bpf_object__unpin_maps() for bpf_object__pin_maps(). But no bpf_object__unpin() for bpf_object__pin(). Adding the former adds symmetry to the API. It's also convenient for cleanup in application code. It's an API I would've used if it was available for a repro I was writing earlier. Signed-off-by:
Daniel Xu <dxu@dxuuu.xyz> Signed-off-by:
Song Liu <song@kernel.org> Link: https://lore.kernel.org/bpf/b2f9d41da4a350281a0b53a804d11b68327e14e5.1692832478.git.dxu@dxuuu.xyz
-
- 23 Aug, 2023 3 commits
-
-
Alexei Starovoitov authored
Yafang Shao says: ==================== bpf: Fix an issue in verifing allow_ptr_leaks Patch #1: An issue found in our local 6.1 kernel. This issue also exists in bpf-next. Patch #2: Selftess for #1 v1->v2: - Add acked-by from Eduard - Fix build error reported by Alexei ==================== Link: https://lore.kernel.org/r/20230823020703.3790-1-laoar.shao@gmail.comSigned-off-by: -
Yafang Shao authored
- Without prev commit $ tools/testing/selftests/bpf/test_progs --name=tc_bpf #232/1 tc_bpf/tc_bpf_root:OK test_tc_bpf_non_root:PASS:set_cap_bpf_cap_net_admin 0 nsec test_tc_bpf_non_root:PASS:disable_cap_sys_admin 0 nsec 0: R1=ctx(off=0,imm=0) R10=fp0 ; if ((long)(iph + 1) > (long)skb->data_end) 0: (61) r2 = *(u32 *)(r1 +80) ; R1=ctx(off=0,imm=0) R2_w=pkt_end(off=0,imm=0) ; struct iphdr *iph = (void *)(long)skb->data + sizeof(struct ethhdr); 1: (61) r1 = *(u32 *)(r1 +76) ; R1_w=pkt(off=0,r=0,imm=0) ; if ((long)(iph + 1) > (long)skb->data_end) 2: (07) r1 += 34 ; R1_w=pkt(off=34,r=0,imm=0) 3: (b4) w0 = 1 ; R0_w=1 4: (2d) if r1 > r2 goto pc+1 R2 pointer comparison prohibited processed 5 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0 test_tc_bpf_non_root:FAIL:test_tc_bpf__open_and_load unexpected error: -13 #233/2 tc_bpf_non_root:FAIL - With prev commit $ tools/testing/selftests/bpf/test_progs --name=tc_bpf #232/1 tc_bpf/tc_bpf_root:OK #232/2 tc_bpf/tc_bpf_non_root:OK #232 tc_bpf:OK Summary: 1/2 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by:
Yafang Shao <laoar.shao@gmail.com> Link: https://lore.kernel.org/r/20230823020703.3790-3-laoar.shao@gmail.comSigned-off-by:
-
Yafang Shao authored
After we converted the capabilities of our networking-bpf program from cap_sys_admin to cap_net_admin+cap_bpf, our networking-bpf program failed to start. Because it failed the bpf verifier, and the error log is "R3 pointer comparison prohibited". A simple reproducer as follows, SEC("cls-ingress") int ingress(struct __sk_buff *skb) { struct iphdr *iph = (void *)(long)skb->data + sizeof(struct ethhdr); if ((long)(iph + 1) > (long)skb->data_end) return TC_ACT_STOLEN; return TC_ACT_OK; } Per discussion with Yonghong and Alexei [1], comparison of two packet pointers is not a pointer leak. This patch fixes it. Our local kernel is 6.1.y and we expect this fix to be backported to 6.1.y, so stable is CCed. [1]. https://lore.kernel.org/bpf/CAADnVQ+Nmspr7Si+pxWn8zkE7hX-7s93ugwC+94aXSy4uQ9vBg@mail.gmail.com/Suggested-by:Alexei Starovoitov <alexei.starovoitov@gmail.com> Signed-off-by:
Eduard Zingerman <eddyz87@gmail.com> Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20230823020703.3790-2-laoar.shao@gmail.comSigned-off-by:
-
- 22 Aug, 2023 6 commits
-
-
Hao Luo authored
I hit a memory leak when testing bpf_program__set_attach_target(). Basically, set_attach_target() may allocate btf_vmlinux, for example, when setting attach target for bpf_iter programs. But btf_vmlinux is freed only in bpf_object_load(), which means if we only open bpf object but not load it, setting attach target may leak btf_vmlinux. So let's free btf_vmlinux in bpf_object__close() anyway. Signed-off-by:
Hao Luo <haoluo@google.com> Signed-off-by:
-
Alexei Starovoitov authored
Kumar Kartikeya Dwivedi says: ==================== Fix for check_func_arg_reg_off Remove a leftover hunk in check_func_arg_reg_off that incorrectly bypasses reg->off == 0 requirement for release kfuncs and helpers. ==================== Link: https://lore.kernel.org/r/20230822175140.1317749-1-memxor@gmail.comSigned-off-by:
-
Kumar Kartikeya Dwivedi authored
Add a selftest for the fix provided in the previous commit. Without the fix, the selftest passes the verifier while it should fail. The special logic for detecting graph root or node for reg->off and bypassing reg->off == 0 guarantee for release helpers/kfuncs has been dropped. Signed-off-by:
Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20230822175140.1317749-3-memxor@gmail.comSigned-off-by:
-
Kumar Kartikeya Dwivedi authored
The commit being fixed introduced a hunk into check_func_arg_reg_off that bypasses reg->off == 0 enforcement when offset points to a graph node or root. This might possibly be done for treating bpf_rbtree_remove and others as KF_RELEASE and then later check correct reg->off in helper argument checks. But this is not the case, those helpers are already not KF_RELEASE and permit non-zero reg->off and verify it later to match the subobject in BTF type. However, this logic leads to bpf_obj_drop permitting free of register arguments with non-zero offset when they point to a graph root or node within them, which is not ok. For instance: struct foo { int i; int j; struct bpf_rb_node node; }; struct foo *f = bpf_obj_new(typeof(*f)); if (!f) ... bpf_obj_drop(f); // OK bpf_obj_drop(&f->i); // still ok from verifier PoV bpf_obj_drop(&f->node); // Not OK, but permitted right now Fix this by dropping the whole part of code altogether. Fixes: 6a3cd331 ("bpf: Migrate release_on_unlock logic to non-owning ref semantics") Signed-off-by: -
Yonghong Song authored
For a bpf_kptr_xchg() with local kptr, if the map value kptr type and allocated local obj type does not match, with the previous patch, the below verifier error message will be logged: R2 is of type <allocated local obj type> but <map value kptr type> is expected Without the previous patch, the test will have unexpected success. Signed-off-by:
-
Yonghong Song authored
When reviewing local percpu kptr support, Alexei discovered a bug wherea bpf_kptr_xchg() may succeed even if the map value kptr type and locally allocated obj type do not match ([1]). Missed struct btf_id comparison is the reason for the bug. This patch added such struct btf_id comparison and will flag verification failure if types do not match. [1] https://lore.kernel.org/bpf/20230819002907.io3iphmnuk43xblu@macbook-pro-8.dhcp.thefacebook.com/#tReported-by:
-
- 21 Aug, 2023 9 commits
-
-
Alexei Starovoitov authored
Jiri Olsa says: ==================== bpf: Add multi uprobe link hi, this patchset is adding support to attach multiple uprobes and usdt probes through new uprobe_multi link. The current uprobe is attached through the perf event and attaching many uprobes takes a lot of time because of that. The main reason is that we need to install perf event for each probed function and profile shows perf event installation (perf_install_in_context) as culprit. The new uprobe_multi link just creates raw uprobes and attaches the bpf program to them without perf event being involved. In addition to being faster we also save file descriptors. For the current uprobe attach we use extra perf event fd for each probed function. The new link just need one fd that covers all the functions we are attaching to. v7 changes: - fixed task release on error path and re-org the error path to be more straightforward [Yonghong] - re-organized uprobe_prog_run locking to follow general pattern and removed might_fault check as it's not needed in uprobe/task context [Yonghong] There's support for bpftrace [2] and tetragon [1]. Also available at: https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git uprobe_multi thanks, jirka [1] https://github.com/cilium/tetragon/pull/936 [2] https://github.com/iovisor/bpftrace/compare/master...olsajiri:bpftrace:uprobe_multi [3] https://lore.kernel.org/bpf/20230628115329.248450-1-laoar.shao@gmail.com/ --- ==================== Link: https://lore.kernel.org/r/20230809083440.3209381-1-jolsa@kernel.orgSigned-off-by: -
Jiri Olsa authored
Attaching extra program to same functions system wide for api and link tests. This way we can test the pid filter works properly when there's extra system wide consumer on the same uprobe that will trigger the original uprobe handler. We expect to have the same counts as before. Signed-off-by:
Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/r/20230809083440.3209381-29-jolsa@kernel.orgSigned-off-by:
-
Jiri Olsa authored
Running api and link tests also with pid filter and checking the probe gets executed only for specific pid. Spawning extra process to trigger attached uprobes and checking we get correct counts from executed programs. Signed-off-by:
-
Jiri Olsa authored
Adding test for cookies setup/retrieval in uprobe_link uprobes and making sure bpf_get_attach_cookie works properly. Signed-off-by:
-
Jiri Olsa authored
Adding test that attaches 50k usdt probes in usdt_multi binary. After the attach is done we run the binary and make sure we get proper amount of hits. With current uprobes: # perf stat --null ./test_progs -n 254/6 #254/6 uprobe_multi_test/bench_usdt:OK #254 uprobe_multi_test:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED Performance counter stats for './test_progs -n 254/6': 1353.659680562 seconds time elapsed With uprobe_multi link: # perf stat --null ./test_progs -n 254/6 #254/6 uprobe_multi_test/bench_usdt:OK #254 uprobe_multi_test:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED Performance counter stats for './test_progs -n 254/6': 0.322046364 seconds time elapsed Signed-off-by: -
Jiri Olsa authored
Adding code in uprobe_multi test binary that defines 50k usdts and will serve as attach point for uprobe_multi usdt bench test in following patch. Signed-off-by:
-
Jiri Olsa authored
Adding test that attaches 50k uprobes in uprobe_multi binary. After the attach is done we run the binary and make sure we get proper amount of hits. The resulting attach/detach times on my setup: test_bench_attach_uprobe:PASS:uprobe_multi__open 0 nsec test_bench_attach_uprobe:PASS:uprobe_multi__attach 0 nsec test_bench_attach_uprobe:PASS:uprobes_count 0 nsec test_bench_attach_uprobe: attached in 0.346s test_bench_attach_uprobe: detached in 0.419s #262/5 uprobe_multi_test/bench_uprobe:OK Signed-off-by:
-
Jiri Olsa authored
Adding uprobe_multi test program that defines 50k uprobe_multi_func_* functions and will serve as attach point for uprobe_multi bench test in following patch. Signed-off-by:
-
Jiri Olsa authored
Adding uprobe_multi test for bpf_link_create attach function. Testing attachment using the struct bpf_link_create_opts. Signed-off-by:
-
