- 27 Jan, 2020 5 commits
-
-
John Fastabend authored
Now that we depend on rcu_call() and synchronize_rcu() to also wait for preempt_disabled region to complete the rcu read critical section in __dev_map_flush() is no longer required. Except in a few special cases in drivers that need it for other reasons. These originally ensured the map reference was safe while a map was also being free'd. And additionally that bpf program updates via ndo_bpf did not happen while flush updates were in flight. But flush by new rules can only be called from preempt-disabled NAPI context. The synchronize_rcu from the map free path and the rcu_call from the delete path will ensure the reference there is safe. So lets remove the rcu_read_lock and rcu_read_unlock pair to avoid any confusion around how this is being protected. If the rcu_read_lock was required it would mean errors in the above logic and the original patch would also be wrong. Now that we have done above we put the rcu_read_lock in the driver code where it is needed in a driver dependent way. I think this helps readability of the code so we know where and why we are taking read locks. Most drivers will not need rcu_read_locks here and further XDP drivers already have rcu_read_locks in their code paths for reading xdp programs on RX side so this makes it symmetric where we don't have half of rcu critical sections define in driver and the other half in devmap. Signed-off-by:
John Fastabend <john.fastabend@gmail.com> Signed-off-by:
Daniel Borkmann <daniel@iogearbox.net> Acked-by:
Jesper Dangaard Brouer <brouer@redhat.com> Link: https://lore.kernel.org/bpf/1580084042-11598-4-git-send-email-john.fastabend@gmail.com
-
John Fastabend authored
virtio_net currently relies on rcu critical section to access the xdp program in its xdp_xmit handler. However, the pointer to the xdp program is only used to do a NULL pointer comparison to determine if xdp is enabled or not. Use rcu_access_pointer() instead of rcu_dereference() to reflect this. Then later when we drop rcu_read critical section virtio_net will not need in special handling. Signed-off-by:
-
John Fastabend authored
Now that we rely on synchronize_rcu and call_rcu waiting to exit perempt-disable regions (NAPI) lets update the comments to reflect this. Fixes: 0536b852 ("xdp: Simplify devmap cleanup") Signed-off-by:
Björn Töpel <bjorn.topel@intel.com> Acked-by:
Song Liu <songliubraving@fb.com> Link: https://lore.kernel.org/bpf/1580084042-11598-2-git-send-email-john.fastabend@gmail.com
-
Vasily Averin authored
If seq_file .next fuction does not change position index, read after some lseek can generate an unexpected output. See also: https://bugzilla.kernel.org/show_bug.cgi?id=206283 v1 -> v2: removed missed increment in end of function Signed-off-by:
Vasily Averin <vvs@virtuozzo.com> Signed-off-by:
-
Andrey Ignatov authored
tools/testing/selftests/bpf/Makefile supports overriding clang, llc and other tools so that custom ones can be used instead of those from PATH. It's convinient and heavily used by some users. Apply same rules to runqslower/Makefile. Signed-off-by:
Andrey Ignatov <rdna@fb.com> Signed-off-by:
Andrii Nakryiko <andriin@fb.com> Link: https://lore.kernel.org/bpf/20200124224142.1833678-1-rdna@fb.com
-
- 25 Jan, 2020 4 commits
-
-
Alexei Starovoitov authored
Jiri Olsa says: ==================== hi, sending 2 fixes to fix kernel support for loading trampoline programs in bcc/bpftrace and allow to unwind through trampoline/dispatcher. Original rfc post [1]. Speedup output of perf bench while running klockstat.py on kprobes vs trampolines: Without: $ perf bench sched messaging -l 50000 ... Total time: 18.571 [sec] With current kprobe tracing: $ perf bench sched messaging -l 50000 ... Total time: 183.395 [sec] With kfunc tracing: $ perf bench sched messaging -l 50000 ... Total time: 39.773 [sec] v4 changes: - rebased on latest bpf-next/master - removed image tree mutex and use trampoline_mutex instead - checking directly for string pointer in patch 1 [Alexei] - skipped helpers patches, as they are no longer needed [Alexei] v3 changes: - added ack from John Fastabend for patch 1 - move out is_bpf_image_address from is_bpf_text_address call [David] v2 changes: - make the unwind work for dispatcher as well - added test for allowed trampolines count - used raw tp pt_regs nest-arrays for trampoline helpers thanks, jirka [1] https://lore.kernel.org/netdev/20191229143740.29143-1-jolsa@kernel.org/ ==================== Signed-off-by:Alexei Starovoitov <ast@kernel.org>
-
Jiri Olsa authored
There's limit of 40 programs tht can be attached to trampoline for one function. Adding test that tries to attach that many plus one extra that needs to fail. Signed-off-by:
Jiri Olsa <jolsa@kernel.org> Signed-off-by:
-
Jiri Olsa authored
When unwinding the stack we need to identify each address to successfully continue. Adding latch tree to keep trampolines for quick lookup during the unwind. The patch uses first 48 bytes for latch tree node, leaving 4048 bytes from the rest of the page for trampoline or dispatcher generated code. It's still enough not to affect trampoline and dispatcher progs maximum counts. Signed-off-by:
-
Jiri Olsa authored
When accessing the context we allow access to arguments with scalar type and pointer to struct. But we deny access for pointer to scalar type, which is the case for many functions. Alexei suggested to take conservative approach and allow currently only string pointer access, which is the case for most functions now: Adding check if the pointer is to string type and allow access to it. Suggested-by:
-
- 24 Jan, 2020 9 commits
-
-
Andrii Nakryiko authored
Fix bug requesting invalid size of reallocated array when constructing CO-RE relocation candidate list. This can cause problems if there are many potential candidates and a very fine-grained memory allocator bucket sizes are used. Fixes: ddc7c304 ("libbpf: implement BPF CO-RE offset relocation algorithm") Reported-by:
William Smith <williampsmith@fb.com> Signed-off-by:
Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/bpf/20200124201847.212528-1-andriin@fb.com
-
Andrii Nakryiko authored
Previously, if libbpf failed to resolve CO-RE relocation for some instructions, it would either return error immediately, or, if .relaxed_core_relocs option was set, would replace relocatable offset/imm part of an instruction with a bogus value (-1). Neither approach is good, because there are many possible scenarios where relocation is expected to fail (e.g., when some field knowingly can be missing on specific kernel versions). On the other hand, replacing offset with invalid one can hide programmer errors, if this relocation failue wasn't anticipated. This patch deprecates .relaxed_core_relocs option and changes the approach to always replacing instruction, for which relocation failed, with invalid BPF helper call instruction. For cases where this is expected, BPF program should already ensure that that instruction is unreachable, in which case this invalid instruction is going to be silently ignored. But if instruction wasn't guarded, BPF program will be rejected at verification step with verifier log pointing precisely to the place in assembly where the problem is. Signed-off-by:
Martin KaFai Lau <kafai@fb.com> Link: https://lore.kernel.org/bpf/20200124053837.2434679-1-andriin@fb.com
-
Lorenz Bauer authored
Currently, there is a lot of false positives if a single reuseport test fails. This is because expected_results and the result map are not cleared. Zero both after individual test runs, which fixes the mentioned false positives. Fixes: 91134d84 ("bpf: Test BPF_PROG_TYPE_SK_REUSEPORT") Signed-off-by:
Lorenz Bauer <lmb@cloudflare.com> Signed-off-by:
Jakub Sitnicki <jakub@cloudflare.com> Acked-by:
-
Lorenz Bauer authored
Include the name of the mismatching result in human readable format when reporting an error. The new output looks like the following: unexpected result result: [1, 0, 0, 0, 0, 0] expected: [0, 0, 0, 0, 0, 0] mismatch on DROP_ERR_INNER_MAP (bpf_prog_linum:153) check_results:FAIL:382 Signed-off-by:
-
Lorenz Bauer authored
The reuseport tests currently suffer from a race condition: FIN packets count towards DROP_ERR_SKB_DATA, since they don't contain a valid struct cmd. Tests will spuriously fail depending on whether check_results is called before or after the FIN is processed. Exit the BPF program early if FIN is set. Fixes: 91134d84 ("bpf: Test BPF_PROG_TYPE_SK_REUSEPORT") Signed-off-by:
-
Lorenz Bauer authored
Use a proper temporary file for sendpage tests. This means that running the tests doesn't clutter the working directory, and allows running the test on read-only filesystems. Fixes: 16962b24 ("bpf: sockmap, add selftests") Signed-off-by:
-
Andrii Nakryiko authored
Add printing out BTF_KIND_FUNC's linkage. Signed-off-by:
Toke Høiland-Jørgensen <toke@redhat.com> Link: https://lore.kernel.org/bpf/20200124054317.2459436-1-andriin@fb.com
-
Andrii Nakryiko authored
Detect when bpftool source code changes and trigger rebuild within selftests/bpf Makefile. Also fix few small formatting problems. Signed-off-by:
-
John Sperbeck authored
The 'duration' variable is referenced in the CHECK() macro, and there are some uses of the macro before 'duration' is set. The clang compiler (validly) complains about this. Sample error: .../selftests/bpf/prog_tests/fexit_test.c:23:6: warning: variable 'duration' is uninitialized when used here [-Wuninitialized] if (CHECK(err, "prog_load sched cls", "err %d errno %d\n", err, errno)) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .../selftests/bpf/test_progs.h:134:25: note: expanded from macro 'CHECK' if (CHECK(err, "prog_load sched cls", "err %d errno %d\n", err, errno)) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ _CHECK(condition, tag, duration, format) ^~~~~~~~ Signed-off-by:John Sperbeck <jsperbeck@google.com> Signed-off-by:
Stanislav Fomichev <sdf@google.com> Signed-off-by:
-
- 23 Jan, 2020 7 commits
-
-
Amol Grover authored
head is traversed using hlist_for_each_entry_rcu outside an RCU read-side critical section but under the protection of dtab->index_lock. Hence, add corresponding lockdep expression to silence false-positive lockdep warnings, and harden RCU lists. Fixes: 6f9d451a ("xdp: Add devmap_hash map type for looking up devices by hashed index") Signed-off-by:
Amol Grover <frextrite@gmail.com> Signed-off-by:
-
Heiner Kallweit authored
Convert suitable drivers to use new helper phy_do_ioctl_running. Signed-off-by:
Heiner Kallweit <hkallweit1@gmail.com> Acked-by:
Timur Tabi <timur@kernel.org> Acked-by:
Florian Fainelli <f.fainelli@gmail.com> Signed-off-by:
David S. Miller <davem@davemloft.net>
-
git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-nextDavid S. Miller authored
Alexei Starovoitov says: ==================== pull-request: bpf-next 2020-01-22 The following pull-request contains BPF updates for your *net-next* tree. We've added 92 non-merge commits during the last 16 day(s) which contain a total of 320 files changed, 7532 insertions(+), 1448 deletions(-). The main changes are: 1) function by function verification and program extensions from Alexei. 2) massive cleanup of selftests/bpf from Toke and Andrii. 3) batched bpf map operations from Brian and Yonghong. 4) tcp congestion control in bpf from Martin. 5) bulking for non-map xdp_redirect form Toke. 6) bpf_send_signal_thread helper from Yonghong. ==================== Signed-off-by: -
Alexei Starovoitov authored
Martin KaFai Lau says: ==================== This set adds bpf_cubic.c example. It was separated from the earlier BPF STRUCT_OPS series. Some highlights since the last post: 1. It is based on EricD recent fixes to the kernel tcp_cubic. [1] 2. The bpf jiffies reading helper is inlined by the verifier. Different from the earlier version, it only reads jiffies alone and does not do usecs/jiffies conversion. 3. The bpf .kconfig map is used to read CONFIG_HZ. [1]: https://patchwork.ozlabs.org/cover/1215066/ v3: - Remove __weak from CONFIG_HZ in patch 3. (Andrii) v2: - Move inlining to fixup_bpf_calls() in patch 1. (Daniel) - It is inlined for 64 BITS_PER_LONG and jit_requested as the map_gen_lookup(). Other cases could be considered together with map_gen_lookup() if needed. - Use usec resolution in bictcp_update() calculation in patch 3. usecs_to_jiffies() is then removed(). (Eric) ==================== Signed-off-by:
-
Martin KaFai Lau authored
This patch adds a bpf_cubic example. Some highlights: 1. CONFIG_HZ .kconfig map is used. 2. In bictcp_update(), calculation is changed to use usec resolution (i.e. USEC_PER_JIFFY) instead of using jiffies. Thus, usecs_to_jiffies() is not used in the bpf_cubic.c. 3. In bitctcp_update() [under tcp_friendliness], the original "while (ca->ack_cnt > delta)" loop is changed to the equivalent "ca->ack_cnt / delta" operation. Signed-off-by:
-
Martin KaFai Lau authored
This patch sync uapi bpf.h to tools/. Signed-off-by:
-
Martin KaFai Lau authored
This patch adds a helper to read the 64bit jiffies. It will be used in a later patch to implement the bpf_cubic.c. The helper is inlined for jit_requested and 64 BITS_PER_LONG as the map_gen_lookup(). Other cases could be considered together with map_gen_lookup() if needed. Signed-off-by:
-
- 22 Jan, 2020 9 commits
-
-
Daniel Borkmann authored
Alexei Starovoitov says: ==================== The last few month BPF community has been discussing an approach to call chaining, since exiting bpt_tail_call() mechanism used in production XDP programs has plenty of downsides. The outcome of these discussion was a conclusion to implement dynamic re-linking of BPF programs. Where rootlet XDP program attached to a netdevice can programmatically define a policy of execution of other XDP programs. Such rootlet would be compiled as normal XDP program and provide a number of placeholder global functions which later can be replaced with future XDP programs. BPF trampoline, function by function verification were building blocks towards that goal. The patch 1 is a final building block. It introduces dynamic program extensions. A number of improvements like more flexible function by function verification and better libbpf api will be implemented in future patches. v1->v2: - addressed Andrii's comments - rebase ==================== Signed-off-by: -
Alexei Starovoitov authored
Add program extension tests that build on top of fexit_bpf2bpf tests. Replace three global functions in previously loaded test_pkt_access.c program with three new implementations: int get_skb_len(struct __sk_buff *skb); int get_constant(long val); int get_skb_ifindex(int val, struct __sk_buff *skb, int var); New function return the same results as original only if arguments match. new_get_skb_ifindex() demonstrates that 'skb' argument doesn't have to be first and only argument of BPF program. All normal skb based accesses are available. Signed-off-by:
-
Alexei Starovoitov authored
Add minimal support for program extensions. bpf_object_open_opts() needs to be called with attach_prog_fd = target_prog_fd and BPF program extension needs to have in .c file section definition like SEC("freplace/func_to_be_replaced"). libbpf will search for "func_to_be_replaced" in the target_prog_fd's BTF and will pass it in attach_btf_id to the kernel. This approach works for tests, but more compex use case may need to request function name (and attach_btf_id that kernel sees) to be more dynamic. Such API will be added in future patches. Signed-off-by: -
Alexei Starovoitov authored
Introduce dynamic program extensions. The users can load additional BPF functions and replace global functions in previously loaded BPF programs while these programs are executing. Global functions are verified individually by the verifier based on their types only. Hence the global function in the new program which types match older function can safely replace that corresponding function. This new function/program is called 'an extension' of old program. At load time the verifier uses (attach_prog_fd, attach_btf_id) pair to identify the function to be replaced. The BPF program type is derived from the target program into extension program. Technically bpf_verifier_ops is copied from target program. The BPF_PROG_TYPE_EXT program type is a placeholder. It has empty verifier_ops. The extension program can call the same bpf helper functions as target program. Single BPF_PROG_TYPE_EXT type is used to extend XDP, SKB and all other program types. The verifier allows only one level of replacement. Meaning that the extension program cannot recursively extend an extension. That also means that the maximum stack size is increasing from 512 to 1024 bytes and maximum function nesting level from 8 to 16. The programs don't always consume that much. The stack usage is determined by the number of on-stack variables used by the program. The verifier could have enforced 512 limit for combined original plus extension program, but it makes for difficult user experience. The main use case for extensions is to provide generic mechanism to plug external programs into policy program or function call chaining. BPF trampoline is used to track both fentry/fexit and program extensions because both are using the same nop slot at the beginning of every BPF function. Attaching fentry/fexit to a function that was replaced is not allowed. The opposite is true as well. Replacing a function that currently being analyzed with fentry/fexit is not allowed. The executable page allocated by BPF trampoline is not used by program extensions. This inefficiency will be optimized in future patches. Function by function verification of global function supports scalars and pointer to context only. Hence program extensions are supported for such class of global functions only. In the future the verifier will be extended with support to pointers to structures, arrays with sizes, etc. Signed-off-by:
-
Heiner Kallweit authored
The first batch of driver conversions missed a few cases where we can use phy_do_ioctl too. Signed-off-by:
Andrew Lunn <andrew@lunn.ch> Signed-off-by:
-
Chris Down authored
When trying to compile with CONFIG_DEBUG_INFO_BTF enabled, I got this error: % make -s Failed to generate BTF for vmlinux Try to disable CONFIG_DEBUG_INFO_BTF make[3]: *** [vmlinux] Error 1 Compiling again without -s shows the true error (that pahole is missing), but since this is fatal, we should show the error unconditionally on stderr as well, not silence it using the `info` function. With this patch: % make -s BTF: .tmp_vmlinux.btf: pahole (pahole) is not available Failed to generate BTF for vmlinux Try to disable CONFIG_DEBUG_INFO_BTF make[3]: *** [vmlinux] Error 1 Signed-off-by:Chris Down <chris@chrisdown.name> Signed-off-by:
-
Daniel Díaz authored
During cross-compilation, it was discovered that LDFLAGS and LDLIBS were not being used while building binaries, leading to defaults which were not necessarily correct. OpenEmbedded reported this kind of problem: ERROR: QA Issue: No GNU_HASH in the ELF binary [...], didn't pass LDFLAGS? Signed-off-by:
Daniel Díaz <daniel.diaz@linaro.org> Signed-off-by:
-
Alexei Starovoitov authored
Restore the 'if (env->cur_state)' check that was incorrectly removed during code move. Under memory pressure env->cur_state can be freed and zeroed inside do_check(). Hence the check is necessary. Fixes: 51c39bb1 ("bpf: Introduce function-by-function verification") Reported-by: syzbot+b296579ba5015704d9fa@syzkaller.appspotmail.com Signed-off-by:
-
Alexei Starovoitov authored
Though the second half of trampoline page is unused a task could be preempted in the middle of the first half of trampoline and two updates to trampoline would change the code from underneath the preempted task. Hence wait for tasks to voluntarily schedule or go to userspace. Add similar wait before freeing the trampoline. Fixes: fec56f58 ("bpf: Introduce BPF trampoline") Reported-by:
Jann Horn <jannh@google.com> Signed-off-by:
Paul E. McKenney <paulmck@kernel.org> Link: https://lore.kernel.org/bpf/20200121032231.3292185-1-ast@kernel.org
-
- 21 Jan, 2020 6 commits
-
-
Björn Töpel authored
XDP sockets use the default implementation of struct sock's sk_data_ready callback, which is sock_def_readable(). This function is called in the XDP socket fast-path, and involves a retpoline. By letting sock_def_readable() have external linkage, and being called directly, the retpoline can be avoided. Signed-off-by:
-
Al Viro authored
kernel/bpf/inode.c misuses kern_path...() - it's much simpler (and more efficient, on top of that) to use user_path...() counterparts rather than bothering with doing getname() manually. Signed-off-by:
Al Viro <viro@zeniv.linux.org.uk> Signed-off-by:
-
git://git.kernel.org/pub/scm/linux/kernel/git/klassert/ipsec-nextDavid S. Miller authored
Steffen Klassert says: ==================== pull request (net-next): ipsec-next 2020-01-21 1) Add support for TCP encapsulation of IKE and ESP messages, as defined by RFC 8229. Patchset from Sabrina Dubroca. Please note that there is a merge conflict in: net/unix/af_unix.c between commit: 3c32da19 ("unix: Show number of pending scm files of receive queue in fdinfo") from the net-next tree and commit: b50b0580 ("net: add queue argument to __skb_wait_for_more_packets and __skb_{,try_}recv_datagram") from the ipsec-next tree. The conflict can be solved as done in linux-next. Please pull or let me know if there are problems. ==================== Signed-off-by:
-
Chen Zhou authored
Fixes coccicheck warning: ./drivers/net/ethernet/amd/declance.c:611:14-15: WARNING comparing pointer to 0 Replace "skb == 0" with "!skb". Signed-off-by:
Chen Zhou <chenzhou10@huawei.com> Signed-off-by:
-
Alex Shi authored
After commit 079096f1 ("tcp/dccp: install syn_recv requests into ehash table") the macro isn't used anymore. remove it. Signed-off-by:
Alex Shi <alex.shi@linux.alibaba.com> Cc: "David S. Miller" <davem@davemloft.net> Cc: Alexey Kuznetsov <kuznet@ms2.inr.ac.ru> Cc: Hideaki YOSHIFUJI <yoshfuji@linux-ipv6.org> Cc: netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by:
-
Alex Shi authored
It's never used after introduced. So maybe better to remove. Signed-off-by:
-
