- 24 Aug, 2023 3 commits
-
-
Andrii Nakryiko authored
Extracting btf_int_encoding() is only meaningful for BTF_KIND_INT, so we need to check that first before inferring signedness. Closes: https://github.com/libbpf/libbpf/issues/704Reported-by:
Lorenz Bauer <lmb@isovalent.com> Signed-off-by:
Andrii Nakryiko <andrii@kernel.org> Acked-by:
Yonghong Song <yonghong.song@linux.dev> Link: https://lore.kernel.org/r/20230824000016.2658017-2-andrii@kernel.orgSigned-off-by:
Martin KaFai Lau <martin.lau@kernel.org>
-
Andrii Nakryiko authored
It seems like it was forgotten to add uprobe_multi binary to .gitignore. Fix this trivial omission. Signed-off-by:
-
Daniel Xu authored
For bpf_object__pin_programs() there is bpf_object__unpin_programs(). Likewise bpf_object__unpin_maps() for bpf_object__pin_maps(). But no bpf_object__unpin() for bpf_object__pin(). Adding the former adds symmetry to the API. It's also convenient for cleanup in application code. It's an API I would've used if it was available for a repro I was writing earlier. Signed-off-by:
Daniel Xu <dxu@dxuuu.xyz> Signed-off-by:
Song Liu <song@kernel.org> Link: https://lore.kernel.org/bpf/b2f9d41da4a350281a0b53a804d11b68327e14e5.1692832478.git.dxu@dxuuu.xyz
-
- 23 Aug, 2023 3 commits
-
-
Alexei Starovoitov authored
Yafang Shao says: ==================== bpf: Fix an issue in verifing allow_ptr_leaks Patch #1: An issue found in our local 6.1 kernel. This issue also exists in bpf-next. Patch #2: Selftess for #1 v1->v2: - Add acked-by from Eduard - Fix build error reported by Alexei ==================== Link: https://lore.kernel.org/r/20230823020703.3790-1-laoar.shao@gmail.comSigned-off-by:Alexei Starovoitov <ast@kernel.org>
-
Yafang Shao authored
- Without prev commit $ tools/testing/selftests/bpf/test_progs --name=tc_bpf #232/1 tc_bpf/tc_bpf_root:OK test_tc_bpf_non_root:PASS:set_cap_bpf_cap_net_admin 0 nsec test_tc_bpf_non_root:PASS:disable_cap_sys_admin 0 nsec 0: R1=ctx(off=0,imm=0) R10=fp0 ; if ((long)(iph + 1) > (long)skb->data_end) 0: (61) r2 = *(u32 *)(r1 +80) ; R1=ctx(off=0,imm=0) R2_w=pkt_end(off=0,imm=0) ; struct iphdr *iph = (void *)(long)skb->data + sizeof(struct ethhdr); 1: (61) r1 = *(u32 *)(r1 +76) ; R1_w=pkt(off=0,r=0,imm=0) ; if ((long)(iph + 1) > (long)skb->data_end) 2: (07) r1 += 34 ; R1_w=pkt(off=34,r=0,imm=0) 3: (b4) w0 = 1 ; R0_w=1 4: (2d) if r1 > r2 goto pc+1 R2 pointer comparison prohibited processed 5 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0 test_tc_bpf_non_root:FAIL:test_tc_bpf__open_and_load unexpected error: -13 #233/2 tc_bpf_non_root:FAIL - With prev commit $ tools/testing/selftests/bpf/test_progs --name=tc_bpf #232/1 tc_bpf/tc_bpf_root:OK #232/2 tc_bpf/tc_bpf_non_root:OK #232 tc_bpf:OK Summary: 1/2 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by:
Yafang Shao <laoar.shao@gmail.com> Link: https://lore.kernel.org/r/20230823020703.3790-3-laoar.shao@gmail.comSigned-off-by:
-
Yafang Shao authored
After we converted the capabilities of our networking-bpf program from cap_sys_admin to cap_net_admin+cap_bpf, our networking-bpf program failed to start. Because it failed the bpf verifier, and the error log is "R3 pointer comparison prohibited". A simple reproducer as follows, SEC("cls-ingress") int ingress(struct __sk_buff *skb) { struct iphdr *iph = (void *)(long)skb->data + sizeof(struct ethhdr); if ((long)(iph + 1) > (long)skb->data_end) return TC_ACT_STOLEN; return TC_ACT_OK; } Per discussion with Yonghong and Alexei [1], comparison of two packet pointers is not a pointer leak. This patch fixes it. Our local kernel is 6.1.y and we expect this fix to be backported to 6.1.y, so stable is CCed. [1]. https://lore.kernel.org/bpf/CAADnVQ+Nmspr7Si+pxWn8zkE7hX-7s93ugwC+94aXSy4uQ9vBg@mail.gmail.com/Suggested-by:Alexei Starovoitov <alexei.starovoitov@gmail.com> Signed-off-by:
Eduard Zingerman <eddyz87@gmail.com> Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20230823020703.3790-2-laoar.shao@gmail.comSigned-off-by:
-
- 22 Aug, 2023 6 commits
-
-
Hao Luo authored
I hit a memory leak when testing bpf_program__set_attach_target(). Basically, set_attach_target() may allocate btf_vmlinux, for example, when setting attach target for bpf_iter programs. But btf_vmlinux is freed only in bpf_object_load(), which means if we only open bpf object but not load it, setting attach target may leak btf_vmlinux. So let's free btf_vmlinux in bpf_object__close() anyway. Signed-off-by:
Hao Luo <haoluo@google.com> Signed-off-by:
-
Alexei Starovoitov authored
Kumar Kartikeya Dwivedi says: ==================== Fix for check_func_arg_reg_off Remove a leftover hunk in check_func_arg_reg_off that incorrectly bypasses reg->off == 0 requirement for release kfuncs and helpers. ==================== Link: https://lore.kernel.org/r/20230822175140.1317749-1-memxor@gmail.comSigned-off-by:
-
Kumar Kartikeya Dwivedi authored
Add a selftest for the fix provided in the previous commit. Without the fix, the selftest passes the verifier while it should fail. The special logic for detecting graph root or node for reg->off and bypassing reg->off == 0 guarantee for release helpers/kfuncs has been dropped. Signed-off-by:
Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20230822175140.1317749-3-memxor@gmail.comSigned-off-by:
-
Kumar Kartikeya Dwivedi authored
The commit being fixed introduced a hunk into check_func_arg_reg_off that bypasses reg->off == 0 enforcement when offset points to a graph node or root. This might possibly be done for treating bpf_rbtree_remove and others as KF_RELEASE and then later check correct reg->off in helper argument checks. But this is not the case, those helpers are already not KF_RELEASE and permit non-zero reg->off and verify it later to match the subobject in BTF type. However, this logic leads to bpf_obj_drop permitting free of register arguments with non-zero offset when they point to a graph root or node within them, which is not ok. For instance: struct foo { int i; int j; struct bpf_rb_node node; }; struct foo *f = bpf_obj_new(typeof(*f)); if (!f) ... bpf_obj_drop(f); // OK bpf_obj_drop(&f->i); // still ok from verifier PoV bpf_obj_drop(&f->node); // Not OK, but permitted right now Fix this by dropping the whole part of code altogether. Fixes: 6a3cd331 ("bpf: Migrate release_on_unlock logic to non-owning ref semantics") Signed-off-by: -
Yonghong Song authored
For a bpf_kptr_xchg() with local kptr, if the map value kptr type and allocated local obj type does not match, with the previous patch, the below verifier error message will be logged: R2 is of type <allocated local obj type> but <map value kptr type> is expected Without the previous patch, the test will have unexpected success. Signed-off-by:
-
Yonghong Song authored
When reviewing local percpu kptr support, Alexei discovered a bug wherea bpf_kptr_xchg() may succeed even if the map value kptr type and locally allocated obj type do not match ([1]). Missed struct btf_id comparison is the reason for the bug. This patch added such struct btf_id comparison and will flag verification failure if types do not match. [1] https://lore.kernel.org/bpf/20230819002907.io3iphmnuk43xblu@macbook-pro-8.dhcp.thefacebook.com/#tReported-by:
-
- 21 Aug, 2023 28 commits
-
-
Alexei Starovoitov authored
Jiri Olsa says: ==================== bpf: Add multi uprobe link hi, this patchset is adding support to attach multiple uprobes and usdt probes through new uprobe_multi link. The current uprobe is attached through the perf event and attaching many uprobes takes a lot of time because of that. The main reason is that we need to install perf event for each probed function and profile shows perf event installation (perf_install_in_context) as culprit. The new uprobe_multi link just creates raw uprobes and attaches the bpf program to them without perf event being involved. In addition to being faster we also save file descriptors. For the current uprobe attach we use extra perf event fd for each probed function. The new link just need one fd that covers all the functions we are attaching to. v7 changes: - fixed task release on error path and re-org the error path to be more straightforward [Yonghong] - re-organized uprobe_prog_run locking to follow general pattern and removed might_fault check as it's not needed in uprobe/task context [Yonghong] There's support for bpftrace [2] and tetragon [1]. Also available at: https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git uprobe_multi thanks, jirka [1] https://github.com/cilium/tetragon/pull/936 [2] https://github.com/iovisor/bpftrace/compare/master...olsajiri:bpftrace:uprobe_multi [3] https://lore.kernel.org/bpf/20230628115329.248450-1-laoar.shao@gmail.com/ --- ==================== Link: https://lore.kernel.org/r/20230809083440.3209381-1-jolsa@kernel.orgSigned-off-by: -
Jiri Olsa authored
Attaching extra program to same functions system wide for api and link tests. This way we can test the pid filter works properly when there's extra system wide consumer on the same uprobe that will trigger the original uprobe handler. We expect to have the same counts as before. Signed-off-by:
Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/r/20230809083440.3209381-29-jolsa@kernel.orgSigned-off-by:
-
Jiri Olsa authored
Running api and link tests also with pid filter and checking the probe gets executed only for specific pid. Spawning extra process to trigger attached uprobes and checking we get correct counts from executed programs. Signed-off-by:
-
Jiri Olsa authored
Adding test for cookies setup/retrieval in uprobe_link uprobes and making sure bpf_get_attach_cookie works properly. Signed-off-by:
-
Jiri Olsa authored
Adding test that attaches 50k usdt probes in usdt_multi binary. After the attach is done we run the binary and make sure we get proper amount of hits. With current uprobes: # perf stat --null ./test_progs -n 254/6 #254/6 uprobe_multi_test/bench_usdt:OK #254 uprobe_multi_test:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED Performance counter stats for './test_progs -n 254/6': 1353.659680562 seconds time elapsed With uprobe_multi link: # perf stat --null ./test_progs -n 254/6 #254/6 uprobe_multi_test/bench_usdt:OK #254 uprobe_multi_test:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED Performance counter stats for './test_progs -n 254/6': 0.322046364 seconds time elapsed Signed-off-by: -
Jiri Olsa authored
Adding code in uprobe_multi test binary that defines 50k usdts and will serve as attach point for uprobe_multi usdt bench test in following patch. Signed-off-by:
-
Jiri Olsa authored
Adding test that attaches 50k uprobes in uprobe_multi binary. After the attach is done we run the binary and make sure we get proper amount of hits. The resulting attach/detach times on my setup: test_bench_attach_uprobe:PASS:uprobe_multi__open 0 nsec test_bench_attach_uprobe:PASS:uprobe_multi__attach 0 nsec test_bench_attach_uprobe:PASS:uprobes_count 0 nsec test_bench_attach_uprobe: attached in 0.346s test_bench_attach_uprobe: detached in 0.419s #262/5 uprobe_multi_test/bench_uprobe:OK Signed-off-by:
-
Jiri Olsa authored
Adding uprobe_multi test program that defines 50k uprobe_multi_func_* functions and will serve as attach point for uprobe_multi bench test in following patch. Signed-off-by:
-
Jiri Olsa authored
Adding uprobe_multi test for bpf_link_create attach function. Testing attachment using the struct bpf_link_create_opts. Signed-off-by:
-
Jiri Olsa authored
Adding uprobe_multi test for bpf_program__attach_uprobe_multi attach function. Testing attachment using glob patterns and via bpf_uprobe_multi_opts paths/syms fields. Signed-off-by:
-
Jiri Olsa authored
Adding uprobe_multi test for skeleton load/attach functions, to test skeleton auto attach for uprobe_multi link. Test that bpf_get_func_ip works properly for uprobe_multi attachment. Signed-off-by:
-
Jiri Olsa authored
We'd like to have single copy of get_time_ns used b bench and test_progs, but we can't just include bench.h, because of conflicting 'struct env' objects. Moving get_time_ns to testing_helpers.h which is being included by both bench and test_progs objects. Signed-off-by:
-
Jiri Olsa authored
Adding support for usdt_manager_attach_usdt to use uprobe_multi link to attach to usdt probes. The uprobe_multi support is detected before the usdt program is loaded and its expected_attach_type is set accordingly. If uprobe_multi support is detected the usdt_manager_attach_usdt gathers uprobes info and calls bpf_program__attach_uprobe to create all needed uprobes. If uprobe_multi support is not detected the old behaviour stays. Also adding usdt.s program section for sleepable usdt probes. Signed-off-by:
-
Jiri Olsa authored
Adding uprobe-multi link detection. It will be used later in bpf_program__attach_usdt function to check and use uprobe_multi link over standard uprobe links. Signed-off-by:
-
Jiri Olsa authored
Adding support for several uprobe_multi program sections to allow auto attach of multi_uprobe programs. Acked-by:
-
Jiri Olsa authored
Adding bpf_program__attach_uprobe_multi function that allows to attach multiple uprobes with uprobe_multi link. The user can specify uprobes with direct arguments: binary_path/func_pattern/pid or with struct bpf_uprobe_multi_opts opts argument fields: const char **syms; const unsigned long *offsets; const unsigned long *ref_ctr_offsets; const __u64 *cookies; User can specify 2 mutually exclusive set of inputs: 1) use only path/func_pattern/pid arguments 2) use path/pid with allowed combinations of: syms/offsets/ref_ctr_offsets/cookies/cnt - syms and offsets are mutually exclusive - ref_ctr_offsets and cookies are optional Any other usage results in error. Signed-off-by: -
Jiri Olsa authored
Adding new uprobe_multi struct to bpf_link_create_opts object to pass multiple uprobe data to link_create attr uapi. Acked-by:
-
Jiri Olsa authored
Adding elf_resolve_pattern_offsets function that looks up offsets for symbols specified by pattern argument. The 'pattern' argument allows wildcards (*?' supported). Offsets are returned in allocated array together with its size and needs to be released by the caller. Signed-off-by:
-
Jiri Olsa authored
Adding elf_resolve_syms_offsets function that looks up offsets for symbols specified in syms array argument. Offsets are returned in allocated array with the 'cnt' size, that needs to be released by the caller. Signed-off-by:
-
Jiri Olsa authored
Adding elf symbol iterator object (and some functions) that follow open-coded iterator pattern and some functions to ease up iterating elf object symbols. The idea is to iterate single symbol section with: struct elf_sym_iter iter; struct elf_sym *sym; if (elf_sym_iter_new(&iter, elf, binary_path, SHT_DYNSYM)) goto error; while ((sym = elf_sym_iter_next(&iter))) { ... } I considered opening the elf inside the iterator and iterate all symbol sections, but then it gets more complicated wrt user checks for when the next section is processed. Plus side is the we don't need 'exit' function, because caller/user is in charge of that. The returned iterated symbol object from elf_sym_iter_next function is placed inside the struct elf_sym_iter, so no extra allocation or argument is needed. Suggested-by: -
Jiri Olsa authored
Adding elf_open/elf_close functions and using it in elf_find_func_offset_from_file function. It will be used in following changes to save some common code. Acked-by:
-
Jiri Olsa authored
Adding new elf object that will contain elf related functions. There's no functional change. Suggested-by:
-
Jiri Olsa authored
Adding new uprobe_multi attach type and link names, so the functions can resolve the new values. Acked-by:
-
Jiri Olsa authored
Adding support for bpf_get_func_ip helper being called from ebpf program attached by uprobe_multi link. It returns the ip of the uprobe. Acked-by:
-
Jiri Olsa authored
Adding support to specify pid for uprobe_multi link and the uprobes are created only for task with given pid value. Using the consumer.filter filter callback for that, so the task gets filtered during the uprobe installation. We still need to check the task during runtime in the uprobe handler, because the handler could get executed if there's another system wide consumer on the same uprobe (thanks Oleg for the insight). Cc: Oleg Nesterov <oleg@redhat.com> Reviewed-by:
Oleg Nesterov <oleg@redhat.com> Signed-off-by:
-
Jiri Olsa authored
Adding support to specify cookies array for uprobe_multi link. The cookies array share indexes and length with other uprobe_multi arrays (offsets/ref_ctr_offsets). The cookies[i] value defines cookie for i-the uprobe and will be returned by bpf_get_attach_cookie helper when called from ebpf program hooked to that specific uprobe. Acked-by:
-
Jiri Olsa authored
Adding new multi uprobe link that allows to attach bpf program to multiple uprobes. Uprobes to attach are specified via new link_create uprobe_multi union: struct { __aligned_u64 path; __aligned_u64 offsets; __aligned_u64 ref_ctr_offsets; __u32 cnt; __u32 flags; } uprobe_multi; Uprobes are defined for single binary specified in path and multiple calling sites specified in offsets array with optional reference counters specified in ref_ctr_offsets array. All specified arrays have length of 'cnt'. The 'flags' supports single bit for now that marks the uprobe as return probe. Acked-by: -
Jiri Olsa authored
Add extra attach_type checks from link_create under bpf_prog_attach_check_attach_type. Suggested-by:
-
