- 06 May, 2020 4 commits
-
-
Luke Nelson authored
This patch optimizes BPF_JSET BPF_K by using a RISC-V andi instruction when the BPF immediate fits in 12 bits, instead of first loading the immediate to a temporary register. Examples of generated code with and without this optimization: BPF_JMP_IMM(BPF_JSET, R1, 2, 1) without optimization: 20: li t1,2 24: and t1,a0,t1 28: bnez t1,0x30 BPF_JMP_IMM(BPF_JSET, R1, 2, 1) with optimization: 20: andi t1,a0,2 24: bnez t1,0x2c BPF_JMP32_IMM(BPF_JSET, R1, 2, 1) without optimization: 20: li t1,2 24: mv t2,a0 28: slli t2,t2,0x20 2c: srli t2,t2,0x20 30: slli t1,t1,0x20 34: srli t1,t1,0x20 38: and t1,t2,t1 3c: bnez t1,0x44 BPF_JMP32_IMM(BPF_JSET, R1, 2, 1) with optimization: 20: andi t1,a0,2 24: bnez t1,0x2c In these examples, because the upper 32 bits of the sign-extended immediate are 0, BPF_JMP BPF_JSET and BPF_JMP32 BPF_JSET are equivalent and therefore the JIT produces identical code for them. Co-developed-by:
Xi Wang <xi.wang@gmail.com> Signed-off-by:
Luke Nelson <luke.r.nels@gmail.com> Signed-off-by:
Daniel Borkmann <daniel@iogearbox.net> Reviewed-by:
Björn Töpel <bjorn.topel@gmail.com> Acked-by:
-
Luke Nelson authored
This patch adds an optimization to BPF_JMP (32- and 64-bit) BPF_K for when the BPF immediate is zero. When the immediate is zero, the code can directly use the RISC-V zero register instead of loading a zero immediate to a temporary register first. Co-developed-by:
-
Luke Nelson authored
This patch adds two optimizations for BPF_ALU BPF_END BPF_FROM_LE in the RV64 BPF JIT. First, it enables the verifier zero-extension optimization to avoid zero extension when imm == 32. Second, it avoids generating code for imm == 64, since it is equivalent to a no-op. Co-developed-by:
-
Luke Nelson authored
Commit 66d0d5a8 ("riscv: bpf: eliminate zero extension code-gen") added support for the verifier zero-extension optimization on RV64 and commit 46dd3d7d ("bpf, riscv: Enable zext optimization for more RV64G ALU ops") enabled it for more instruction cases. However, BPF_LSH BPF_X and BPF_{LSH,RSH,ARSH} BPF_K are still missing the optimization. This patch enables the zero-extension optimization for these remaining cases. Co-developed-by:
-
- 05 May, 2020 1 commit
-
-
Arnd Bergmann authored
The newly added bpf_stats_handler function has the wrong #ifdef check around it, leading to an unused-function warning when CONFIG_SYSCTL is disabled: kernel/sysctl.c:205:12: error: unused function 'bpf_stats_handler' [-Werror,-Wunused-function] static int bpf_stats_handler(struct ctl_table *table, int write, Fix the check to match the reference. Fixes: d46edd67 ("bpf: Sharing bpf runtime stats with BPF_ENABLE_STATS") Signed-off-by:
Arnd Bergmann <arnd@arndb.de> Signed-off-by:
Alexei Starovoitov <ast@kernel.org> Reviewed-by:
Luis Chamberlain <mcgrof@kernel.org> Acked-by:
Martin KaFai Lau <kafai@fb.com> Acked-by:
Song Liu <songliubraving@fb.com> Link: https://lore.kernel.org/bpf/20200505140734.503701-1-arnd@arndb.de
-
- 04 May, 2020 5 commits
-
-
Magnus Karlsson authored
Remove the unnecessary member of address in struct xdp_umem as it is only used during the umem registration. No need to carry this around as it is not used during run-time nor when unregistering the umem. Signed-off-by:
Magnus Karlsson <magnus.karlsson@intel.com> Signed-off-by:
Jonathan Lemon <jonathan.lemon@gmail.com> Link: https://lore.kernel.org/bpf/1588599232-24897-3-git-send-email-magnus.karlsson@intel.com
-
Magnus Karlsson authored
Change two variables names so that it is clearer what they represent. The first one is xsk_list that in fact only contains the list of AF_XDP sockets with a Tx component. Change this to xsk_tx_list for improved clarity. The second variable is size in the ring structure. One might think that this is the size of the ring, but it is in fact the size of the umem, copied into the ring structure to improve performance. Rename this variable umem_size to avoid any confusion. Signed-off-by:
-
Arnd Bergmann authored
gcc-10 warns about accesses to zero-length arrays: kernel/bpf/core.c: In function 'bpf_patch_insn_single': cc1: warning: writing 8 bytes into a region of size 0 [-Wstringop-overflow=] In file included from kernel/bpf/core.c:21: include/linux/filter.h:550:20: note: at offset 0 to object 'insnsi' with size 0 declared here 550 | struct bpf_insn insnsi[0]; | ^~~~~~ In this case, we really want to have two flexible-array members, but that is not possible. Removing the union to make insnsi a flexible-array member while leaving insns as a zero-length array fixes the warning, as nothing writes to the other one in that way. This trick only works on linux-3.18 or higher, as older versions had additional members in the union. Fixes: 60a3b225 ("net: bpf: make eBPF interpreter images read-only") Signed-off-by: -
Luke Nelson authored
This patch adds an optimization that uses the asr immediate instruction for BPF_ALU BPF_ARSH BPF_K, rather than loading the immediate to a temporary register. This is similar to existing code for handling BPF_ALU BPF_{LSH,RSH} BPF_K. This optimization saves two instructions and is more consistent with LSH and RSH. Example of the code generated for BPF_ALU32_IMM(BPF_ARSH, BPF_REG_0, 5) before the optimization: 2c: mov r8, #5 30: mov r9, #0 34: asr r0, r0, r8 and after optimization: 2c: asr r0, r0, #5 Tested on QEMU using lib/test_bpf and test_verifier. Co-developed-by: -
Luke Nelson authored
This patch optimizes the code generated by emit_a32_arsh_r64, which handles the BPF_ALU64 BPF_ARSH BPF_X instruction. The original code uses a conditional B followed by an unconditional ORR. The optimization saves one instruction by removing the B instruction and using a conditional ORR (with an inverted condition). Example of the code generated for BPF_ALU64_REG(BPF_ARSH, BPF_REG_0, BPF_REG_1), before optimization: 34: rsb ip, r2, #32 38: subs r9, r2, #32 3c: lsr lr, r0, r2 40: orr lr, lr, r1, lsl ip 44: bmi 0x4c 48: orr lr, lr, r1, asr r9 4c: asr ip, r1, r2 50: mov r0, lr 54: mov r1, ip and after optimization: 34: rsb ip, r2, #32 38: subs r9, r2, #32 3c: lsr lr, r0, r2 40: orr lr, lr, r1, lsl ip 44: orrpl lr, lr, r1, asr r9 48: asr ip, r1, r2 4c: mov r0, lr 50: mov r1, ip Tested on QEMU using lib/test_bpf and test_verifier. Co-developed-by:
-
- 03 May, 2020 30 commits
-
-
David S. Miller authored
Karsten Graul says: ==================== net/smc: add and delete link processing These patches add the 'add link' and 'delete link' processing as SMC server and client. This processing allows to establish and remove links of a link group dynamically. v2: Fix mess up with unused static functions. Merge patch 8 into patch 4. Postpone patch 13 to next series. ==================== Signed-off-by:David S. Miller <davem@davemloft.net>
-
Karsten Graul authored
As SMC server, when a second link was deleted, trigger the setup of an asymmetric link. Do this by enqueueing a local ADD_LINK message which is processed by the LLC layer as if it were received from peer. Do the same when a new IB port became active and a new link could be created. smc_llc_srv_add_link_local() enqueues a local ADD_LINK message. And smc_llc_srv_delete_link_local() is used the same way to enqueue a local DELETE_LINK message. This is used when an IB port is no longer active. Signed-off-by:
Karsten Graul <kgraul@linux.ibm.com> Reviewed-by:
Ursula Braun <ubraun@linux.ibm.com> Signed-off-by:
-
Karsten Graul authored
Add smc_llc_process_srv_delete_link() to process a DELETE_LINK request as SMC server. When the request is to delete ALL links then terminate the whole link group. If not, find the link to delete by its link_id, send the DELETE_LINK request LLC message and wait for the response. No matter if a response was received, clear the deleted link and update the link group state. Signed-off-by:
-
Karsten Graul authored
Add smc_llc_process_cli_delete_link() to process a DELETE_LINK request as SMC client. When the request is to delete ALL links then terminate the whole link group. If not, find the link to delete by its link_id, send the DELETE_LINK response LLC message and then clear the deleted link. Finally determine and update the link group state. Signed-off-by:
-
Karsten Graul authored
Introduce a work that is scheduled when a new DELETE_LINK LLC request is received. The work will call either the SMC client or SMC server DELETE_LINK processing. And use the LLC flow framework to process incoming DELETE_LINK LLC messages, scheduling the llc_del_link_work for those events. With these changes smc_lgr_forget() is only called by one function and can be migrated into smc_lgr_cleanup_early(). Signed-off-by:
-
Karsten Graul authored
When a link group moved from asymmetric to symmetric state then the dangling asymmetric link can be deleted. Add smc_llc_find_asym_link() to find the respective link and add smc_llc_delete_asym_link() to delete it. Signed-off-by:
-
Karsten Graul authored
This patch finalizes the ADD_LINK processing of new links. Send the CONFIRM_LINK request to the peer, receive the response and set link state to ACTIVE. Signed-off-by:
-
Karsten Graul authored
Part of SMC server new link establishment is the exchange of rkeys for used buffers. Loop over all used RMB buffers and send ADD_LINK_CONTINUE LLC messages to the peer. Signed-off-by:
-
Karsten Graul authored
First set of functions to process an ADD_LINK LLC request as an SMC server. Find an alternate IB device, determine the new link group type and get the index for the new link. Then initialize the link and send the ADD_LINK LLC message to the peer. Save the contents of the response, ready the link, map all used buffers and register the buffers with the IB device. If any error occurs, stop the processing and clear the link. And call smc_llc_srv_add_link() in af_smc.c to start second link establishment after the initial link of a link group was created. Signed-off-by:
-
Karsten Graul authored
This patch finalizes the ADD_LINK processing of new links. Receive the CONFIRM_LINK request from peer, complete the link initialization, register all used buffers with the IB device and finally send the CONFIRM_LINK response, which completes the ADD_LINK processing. And activate smc_llc_cli_add_link() in af_smc.c. Signed-off-by:
-
Karsten Graul authored
Part of the SMC client new link establishment process is the exchange of rkeys for all used buffers. Add new LLC message type ADD_LINK_CONTINUE which is used to exchange rkeys of all current RMB buffers. Add functions to iterate over all used RMB buffers of the link group, and implement the ADD_LINK_CONTINUE processing. Signed-off-by:
-
Karsten Graul authored
First set of functions to process an ADD_LINK LLC request as an SMC client. Find an alternate IB device, determine the new link group type and get the index for the new link. Then ready the link, map the buffers and send an ADD_LINK LLC response. If any error occurs, send a reject LLC message and terminate the processing. Add smc_llc_alloc_alt_link() to find a free link index for a new link, depending on the new link group type. Signed-off-by:
-
David S. Miller authored
Sameeh Jubran says: ==================== Enhance current features in ena driver Difference from v2: * dropped patch "net: ena: move llq configuration from ena_probe to ena_device_init()" * reworked patch ""net: ena: implement ena_com_get_admin_polling_mode() to drop the prototype Difference from v1: * reodered paches #01 and #02. * dropped adding Rx/Tx drops to ethtool in patch #08 V1: This patchset introduces the following: * minor changes to RSS feature * add total rx and tx drop counter * add unmask_interrupt counter for ethtool statistics * add missing implementation for ena_com_get_admin_polling_mode() * some minor code clean-up and cosmetics * use SHUTDOWN as reset reason when closing interface ==================== Signed-off-by: -
Arthur Kiyanovski authored
Extract code to ena_indirection_table_set() to make the code cleaner. Signed-off-by:
Sameeh Jubran <sameehj@amazon.com> Signed-off-by:
Arthur Kiyanovski <akiyano@amazon.com> Signed-off-by:
-
Sameeh Jubran authored
The macros in ena_com.h have inconsistent spaces between the macro name and it's value. This commit sets all the macros to have a single space between the name and value. Signed-off-by:
-
Sameeh Jubran authored
The 'ENA_REGS_RESET_SHUTDOWN' enum indicates a normal driver shutdown / removal procedure. Also, a comment is added to one of the reset reason assignments for code clarity. Signed-off-by:
Shay Agroskin <shayagr@amazon.com> Signed-off-by:
-
Arthur Kiyanovski authored
Before this commit there was a function prototype named ena_com_get_ena_admin_polling_mode() that was never implemented. This patch simply deletes it. Signed-off-by:
Igor Chauskin <igorch@amazon.com> Signed-off-by:
-
Sameeh Jubran authored
1. Add support for getting tx drops from the device and saving them in the driver. 2. Report tx via netdev stats. Signed-off-by:
Guy Tzalik <gtzalik@amazon.com> Signed-off-by:
-
Sameeh Jubran authored
Add unmask interrupts statistics to ethtool. Signed-off-by:
Netanel Belgazal <netanel@amazon.com> Signed-off-by:
-
Sameeh Jubran authored
Both key and func parameters are pointers on the stack. Setting them to NULL does nothing. The original intent was to leave the key and func unset in this case, but for this to happen nothing needs to be done as the calling function ethtool_get_rxfh() already clears key and func. This commit removes the above described useless code. Signed-off-by:
-
Sameeh Jubran authored
This commit contains 2 cosmetic changes: 1. Use ena_com_check_supported_feature_id() in ena_com_hash_key_fill_default_key() instead of rewriting its implementation. This also saves us a superfluous admin command by using the cached value. 2. Change if conditions in ena_com_rss_init() to be clearer. Signed-off-by:
-
Arthur Kiyanovski authored
Currently in the driver we are setting the hash function to be CRC32. Starting with this commit we want to change the default behaviour so that we set the hash function to be Toeplitz instead. Signed-off-by:
-
Sameeh Jubran authored
Current code does not allow setting the hash function without changing the key. This commit enables it. To achieve this we separate ena_com_get_hash_function() to 2 functions: ena_com_get_hash_function() - which gets only the hash function, and ena_com_get_hash_key() - which gets only the hash key. Also return 0 instead of rc at the end of ena_get_rxfh() since all previous operations succeeded. Signed-off-by:
-
Arthur Kiyanovski authored
In case the "func" parameter is NULL we now return "-EINVAL". This shouldn't happen in general, but when it does happen, this is the proper way to handle it. We also check func for NULL in the beginning of the function, as there is no reason to do all the work and realize in the end of the function it was useless. Signed-off-by:
-
Arthur Kiyanovski authored
Currently when ena_set_hash_function() fails the hash function is restored to the previous value by calling an admin command to get the hash function from the device. In this commit we avoid the admin command, by saving the previous hash function before calling ena_set_hash_function() and using this previous value to restore the hash function in case of failure of ena_set_hash_function(). Signed-off-by:
-
David S. Miller authored
Eric Dumazet says: ==================== net_sched: sch_fq: round of optimizations This series is focused on better layout of struct fq_flow to reduce number of cache line misses in fq_enqueue() and dequeue operations. ==================== Signed-off-by: -
Eric Dumazet authored
The prefetch() done in fq_dequeue() can be done a bit earlier after the refactoring of the code done in the prior patch. Signed-off-by:
Eric Dumazet <edumazet@google.com> Signed-off-by:
-
Eric Dumazet authored
This refactors the code to not call fq_peek() from fq_dequeue_head() since the caller can provide the skb. Also rename fq_dequeue_head() to fq_dequeue_skb() because 'head' is a bit vague, given the skb could come from t_root rb-tree. Signed-off-by:
-
Eric Dumazet authored
fq_gc() already builds a small array of pointers, so using kmem_cache_free_bulk() needs very little change. Signed-off-by:
-
Eric Dumazet authored
sizeof(struct fq_flow) is 112 bytes on 64bit arches. This means that half of them use two cache lines, but 50% use three cache lines. This patch adds cache line alignment, and makes sure that only the first cache line is touched by fq_enqueue(), which is more expensive that fq_dequeue() in general. Signed-off-by:
-
