- 08 Mar, 2024 40 commits
-
-
Ido Schimmel authored
Add nexthop group entry stats to count the number of packets forwarded via each nexthop in the group. The stats will be exposed to user space for better data path observability in the next patch. The per-CPU stats pointer is placed at the beginning of 'struct nh_grp_entry', so that all the fields accessed for the data path reside on the same cache line: struct nh_grp_entry { struct nexthop * nh; /* 0 8 */ struct nh_grp_entry_stats * stats; /* 8 8 */ u8 weight; /* 16 1 */ /* XXX 7 bytes hole, try to pack */ union { struct { atomic_t upper_bound; /* 24 4 */ } hthr; /* 24 4 */ struct { struct list_head uw_nh_entry; /* 24 16 */ u16 count_buckets; /* 40 2 */ u16 wants_buckets; /* 42 2 */ } res; /* 24 24 */ }; /* 24 24 */ struct list_head nh_list; /* 48 16 */ /* --- cacheline 1 boundary (64 bytes) --- */ struct nexthop * nh_parent; /* 64 8 */ /* size: 72, cachelines: 2, members: 6 */ /* sum members: 65, holes: 1, sum holes: 7 */ /* last cacheline: 8 bytes */ }; Co-developed-by:Petr Machata <petrm@nvidia.com> Signed-off-by:
Ido Schimmel <idosch@nvidia.com> Signed-off-by:
David S. Miller <davem@davemloft.net>
-
Petr Machata authored
In order to add per-nexthop statistics, but still not increase netlink message size for consumers that do not care about them, there needs to be a toggle through which the user indicates their desire to get the statistics. To that end, add a new attribute, NHA_OP_FLAGS. The idea is to be able to use the attribute for carrying of arbitrary operation-specific flags, i.e. not make it specific for get / dump. Add the new attribute to get and dump policies, but do not actually allow any flags yet -- those will come later as the flags themselves are defined. Add the necessary parsing code. Signed-off-by:
David Ahern <dsahern@kernel.org> Reviewed-by:
-
Petr Machata authored
A following patch will introduce a new attribute, op-specific flags to adjust the behavior of an operation. Different operations will recognize different flags. - To make the differentiation possible, stop sharing the policies for get and del operations. - To allow querying for presence of the attribute, have all the attribute arrays sized to NHA_MAX, regardless of what is permitted by policy, and pass the corresponding value to nlmsg_parse() as well. Signed-off-by:
-
Sai Krishna authored
This patch adds TC offload support for matching TCP flags from TCP header. Example usage: tc qdisc add dev eth0 ingress TC rule to drop the TCP SYN packets: tc filter add dev eth0 ingress protocol ip flower ip_proto tcp tcp_flags 0x02/0x3f skip_sw action drop Signed-off-by:
Sai Krishna <saikrishnag@marvell.com> Reviewed-by:
Simon Horman <horms@kernel.org> Signed-off-by:
-
fuyuanli authored
It is useful to expose skb addr and sock addr to user in tracepoint tcp_probe, so that we can get more information while monitoring receiving of tcp data, by ebpf or other ways. For example, we need to identify a packet by seq and end_seq when calculate transmit latency between layer 2 and layer 4 by ebpf, but which is not available in tcp_probe, so we can only use kprobe hooking tcp_rcv_established to get them. But we can use tcp_probe directly if skb addr and sock addr are available, which is more efficient. Signed-off-by:
fuyuanli <fuyuanli@didiglobal.com> Reviewed-by:
Jason Xing <kerneljasonxing@gmail.com> Signed-off-by:
-
Jakub Kicinski authored
softnet_data->time_squeeze is sometimes used as a proxy for host overload or indication of scheduling problems. In practice this statistic is very noisy and has hard to grasp units - e.g. is 10 squeezes a second to be expected, or high? Delaying network (NAPI) processing leads to drops on NIC queues but also RTT bloat, impacting pacing and CA decisions. Stalls are a little hard to detect on the Rx side, because there may simply have not been any packets received in given period of time. Packet timestamps help a little bit, but again we don't know if packets are stale because we're not keeping up or because someone (*cough* cgroups) disabled IRQs for a long time. We can, however, use Tx as a proxy for Rx stalls. Most drivers use combined Rx+Tx NAPIs so if Tx gets starved so will Rx. On the Tx side we know exactly when packets get queued, and completed, so there is no uncertainty. This patch adds stall checks to BQL. Why BQL? Because it's a convenient place to add such checks, already called by most drivers, and it has copious free space in its structures (this patch adds no extra cache references or dirtying to the fast path). The algorithm takes one parameter - max delay AKA stall threshold and increments a counter whenever NAPI got delayed for at least that amount of time. It also records the length of the longest stall. To be precise every time NAPI has not polled for at least stall thrs we check if there were any Tx packets queued between last NAPI run and now - stall_thrs/2. Unlike the classic Tx watchdog this mechanism does not ignore stalls caused by Tx being disabled, or loss of link. I don't think the check is worth the complexity, and stall is a stall, whether due to host overload, flow control, link down... doesn't matter much to the application. We have been running this detector in production at Meta for 2 years, with the threshold of 8ms. It's the lowest value where false positives become rare. There's still a constant stream of reported stalls (especially without the ksoftirqd deferral patches reverted), those who like their stall metrics to be 0 may prefer higher value. Signed-off-by:
Jakub Kicinski <kuba@kernel.org> Signed-off-by:
Breno Leitao <leitao@debian.org> Signed-off-by:
-
Colin Ian King authored
The inlined helper function calc_tx_descs is not used and is redundant. Remove it. Cleans up clang scan build warning: drivers/net/ethernet/chelsio/cxgb4/sge.c:814:28: warning: unused function 'calc_tx_descs' [-Wunused-function] Signed-off-by:
Colin Ian King <colin.i.king@gmail.com> Signed-off-by:
-
Jakub Kicinski authored
Jakub Kicinski says: ==================== netdev: add per-queue statistics Per queue stats keep coming up, so it's about time someone laid the foundation. This series adds the uAPI, a handful of stats and a sample support for bnxt. It's not very comprehensive in terms of stat types or driver support. The expectation is that the support will grow organically. If we have the basic pieces in place it will be easy for reviewers to request new stats, or use of the API in place of ethtool -S. See patch 3 for sample output. v2: https://lore.kernel.org/all/20240229010221.2408413-1-kuba@kernel.org/ v1: https://lore.kernel.org/all/20240226211015.1244807-1-kuba@kernel.org/ rfc: https://lore.kernel.org/all/20240222223629.158254-1-kuba@kernel.org/ ==================== Link: https://lore.kernel.org/r/20240306195509.1502746-1-kuba@kernel.orgSigned-off-by:
-
Jakub Kicinski authored
Support per-queue statistics API in bnxt. $ ethtool -S eth0 NIC statistics: [0]: rx_ucast_packets: 1418 [0]: rx_mcast_packets: 178 [0]: rx_bcast_packets: 0 [0]: rx_discards: 0 [0]: rx_errors: 0 [0]: rx_ucast_bytes: 1141815 [0]: rx_mcast_bytes: 16766 [0]: rx_bcast_bytes: 0 [0]: tx_ucast_packets: 1734 ... $ ./cli.py --spec netlink/specs/netdev.yaml \ --dump qstats-get --json '{"scope": "queue"}' [{'ifindex': 2, 'queue-id': 0, 'queue-type': 'rx', 'rx-alloc-fail': 0, 'rx-bytes': 1164931, 'rx-packets': 1641}, ... {'ifindex': 2, 'queue-id': 0, 'queue-type': 'tx', 'tx-bytes': 631494, 'tx-packets': 1771}, ... Reset the per queue counters: $ ethtool -L eth0 combined 4 Inspect again: $ ./cli.py --spec netlink/specs/netdev.yaml \ --dump qstats-get --json '{"scope": "queue"}' [{'ifindex': 2, 'queue-id': 0, 'queue-type': 'rx', 'rx-alloc-fail': 0, 'rx-bytes': 32397, 'rx-packets': 145}, ... {'ifindex': 2, 'queue-id': 0, 'queue-type': 'tx', 'tx-bytes': 37481, 'tx-packets': 196}, ... $ ethtool -S eth0 | head NIC statistics: [0]: rx_ucast_packets: 174 [0]: rx_mcast_packets: 3 [0]: rx_bcast_packets: 0 [0]: rx_discards: 0 [0]: rx_errors: 0 [0]: rx_ucast_bytes: 37151 [0]: rx_mcast_bytes: 267 [0]: rx_bcast_bytes: 0 [0]: tx_ucast_packets: 267 ... Totals are still correct: $ ./cli.py --spec netlink/specs/netdev.yaml --dump qstats-get [{'ifindex': 2, 'rx-alloc-fail': 0, 'rx-bytes': 281949995, 'rx-packets': 216524, 'tx-bytes': 52694905, 'tx-packets': 75546}] $ ip -s link show dev eth0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 link/ether 14:23:f2:61:05:40 brd ff:ff:ff:ff:ff:ff RX: bytes packets errors dropped missed mcast 282519546 218100 0 0 0 516 TX: bytes packets errors dropped carrier collsns 53323054 77674 0 0 0 0 Acked-by:Stanislav Fomichev <sdf@google.com> Reviewed-by:
Amritha Nambiar <amritha.nambiar@intel.com> Reviewed-by:
Michael Chan <michael.chan@broadcom.com> Reviewed-by:
Xuan Zhuo <xuanzhuo@linux.alibaba.com> Link: https://lore.kernel.org/r/20240306195509.1502746-4-kuba@kernel.orgSigned-off-by:
-
Jakub Kicinski authored
Rx alloc failures are commonly counted by drivers. Support reporting those via netdev-genl queue stats. Acked-by:
-
Jakub Kicinski authored
The ethtool-nl family does a good job exposing various protocol related and IEEE/IETF statistics which used to get dumped under ethtool -S, with creative names. Queue stats don't have a netlink API, yet, and remain a lion's share of ethtool -S output for new drivers. Not only is that bad because the names differ driver to driver but it's also bug-prone. Intuitively drivers try to report only the stats for active queues, but querying ethtool stats involves multiple system calls, and the number of stats is read separately from the stats themselves. Worse still when user space asks for values of the stats, it doesn't inform the kernel how big the buffer is. If number of stats increases in the meantime kernel will overflow user buffer. Add a netlink API for dumping queue stats. Queue information is exposed via the netdev-genl family, so add the stats there. Support per-queue and sum-for-device dumps. Latter will be useful when subsequent patches add more interesting common stats than just bytes and packets. The API does not currently distinguish between HW and SW stats. The expectation is that the source of the stats will either not matter much (good packets) or be obvious (skb alloc errors). Acked-by:
-
Jakub Kicinski authored
Eric Dumazet says: ==================== net: group together hot data While our recent structure reorganizations were focused on increasing max throughput, there is still an area where improvements are much needed. In many cases, a cpu handles one packet at a time, instead of a nice batch. Hardware interrupt. -> Software interrupt. -> Network/Protocol stacks. If the cpu was idle or busy in other layers, it has to pull many cache lines. This series adds a new net_hotdata structure, where some critical (and read-mostly) data used in rx and tx path is packed in a small number of cache lines. Synthetic benchmarks will not see much difference, but latency of single packet should improve. net_hodata current size on 64bit is 416 bytes, but might grow in the future. Also move RPS definitions to a new include file. ==================== Link: https://lore.kernel.org/r/20240306160031.874438-1-edumazet@google.comSigned-off-by:
-
Eric Dumazet authored
rps_sock_flow_table and rps_cpu_mask are used in fast path. Move them to net_hotdata for better cache locality. Signed-off-by:
Eric Dumazet <edumazet@google.com> Acked-by:
Soheil Hassas Yeganeh <soheil@google.com> Reviewed-by:
-
Eric Dumazet authored
Move RPS related structures and helpers from include/linux/netdevice.h and include/net/sock.h to a new include file. Signed-off-by:
-
Eric Dumazet authored
Use a 32bit hole in "struct net_offload" to store the remaining 32bit secrets used by TCPv6 and UDPv6. Signed-off-by:
-
Eric Dumazet authored
"struct inet6_protocol" has a 32bit hole in 32bit arches. Use it to store the 32bit secret used by UDP and TCP, to increase cache locality in rx path. Signed-off-by:
-
Eric Dumazet authored
"struct net_protocol" has a 32bit hole in 32bit arches. Use it to store the 32bit secret used by UDP and TCP, to increase cache locality in rx path. Signed-off-by:
-
Eric Dumazet authored
These structures are read in rx path, move them to net_hotdata for better cache locality. Signed-off-by:
-
Eric Dumazet authored
These structures are read in rx path, move them to net_hotdata for better cache locality. Signed-off-by:
-
Eric Dumazet authored
These structures are used in GRO and GSO paths. Move them to net_hodata for better cache locality. v2: udpv6_offload definition depends on CONFIG_INET=y Signed-off-by:
-
Eric Dumazet authored
skbuff_cache, skbuff_fclone_cache and skb_small_head_cache are used in rx/tx fast paths. Move them to net_hotdata for better cache locality. Signed-off-by:
-
Eric Dumazet authored
dev_rx_weight is read from process_backlog(). Move it to net_hotdata for better cache locality. Signed-off-by:
-
Eric Dumazet authored
dev_tx_weight is used in tx fast path. Move it to net_hotdata for better cache locality. Signed-off-by:
-
Eric Dumazet authored
These are used in TCP fast paths. Move them into net_hotdata for better cache locality. v2: tcpv6_offload definition depends on CONFIG_INET Signed-off-by:
-
Eric Dumazet authored
These structures are used in GRO and GSO paths. v2: ipv6_packet_offload definition depends on CONFIG_INET Signed-off-by:
-
Eric Dumazet authored
netdev_max_backlog is used in rx fat path. Move it to net_hodata for better cache locality. Signed-off-by:
-
Eric Dumazet authored
ptype_all is used in rx/tx fast paths. Move it to net_hotdata for better cache locality. Signed-off-by:
-
Eric Dumazet authored
netdev_tstamp_prequeue is used in rx path. Move it to net_hotdata for better cache locality. Signed-off-by:
-
Eric Dumazet authored
netdev_budget and netdev_budget are used in rx path (net_rx_action()) Move them into net_hotdata for better cache locality. Signed-off-by:
-
Eric Dumazet authored
Instead of spreading networking critical fields all over the places, add a custom net_hotdata structure so that we can precisely control its layout. In this first patch, move : - gro_normal_batch used in rx (GRO stack) - offload_base used in rx and tx (GRO and TSO stacks) Signed-off-by:
-
Jakub Kicinski authored
Matthieu Baerts says: ==================== selftests: mptcp: share code and fix shellcheck warnings This series cleans MPTCP selftests code. Patch 1 stops using 'iptables-legacy' if available, but uses 'iptables', which is likely 'iptables-nft' behind. Patches 2, 4 and 6 move duplicated code to mptcp_lib.sh. Patch 3 is a preparation for patch 4, and patch 5 adds generic actions at the creation and deletion of netns. Patches 7 to 11 disable a few shellcheck warnings, and fix the rest, so it is easy to spot real issues later. MPTCP CI is checking that now. Patch 12 avoids redoing some actions at init time twice, e.g. restarting the pm events tool. v1: https://lore.kernel.org/r/20240305-upstream-net-next-20240304-selftests-mptcp-shared-code-shellcheck-v1-0-66618ea5504e@kernel.org ==================== Link: https://lore.kernel.org/r/20240306-upstream-net-next-20240304-selftests-mptcp-shared-code-shellcheck-v2-0-bc79e6e5e6a0@kernel.orgSigned-off-by:
-
Matthieu Baerts (NGI0) authored
'make_connection' is launched twice: once for IPv4, once for IPv6. But then, the "pm_nl_ctl events" was launched a first time, killed, then relaunched after for no particular reason. We can then move this code, and the generation of the temp file to exchange, to the init part, and remove extra conditions that no longer needed. Reviewed-by:
Mat Martineau <martineau@kernel.org> Signed-off-by:
Matthieu Baerts (NGI0) <matttbe@kernel.org> Link: https://lore.kernel.org/r/20240306-upstream-net-next-20240304-selftests-mptcp-shared-code-shellcheck-v2-12-bc79e6e5e6a0@kernel.orgSigned-off-by:
-
Matthieu Baerts (NGI0) authored
shellcheck recently helped to prevent issues. It is then good to fix the other harmless issues in order to spot "real" ones later. Here, two categories of warnings are now ignored: - SC2317: Command appears to be unreachable. The cleanup() function is invoked indirectly via the EXIT trap. - SC2086: Double quote to prevent globbing and word splitting. This is recommended, but the current usage is correct and there is no need to do all these modifications to be compliant with this rule. For the modifications: - SC2034: ksft_skip appears unused. - SC2004: $/${} is unnecessary on arithmetic variables. Now this script is shellcheck (0.9.0) compliant. We can easily spot new issues. Reviewed-by: -
Matthieu Baerts (NGI0) authored
shellcheck recently helped to prevent issues. It is then good to fix the other harmless issues in order to spot "real" ones later. Here, two categories of warnings are now ignored: - SC2317: Command appears to be unreachable. The cleanup() function is invoked indirectly via the EXIT trap. - SC2086: Double quote to prevent globbing and word splitting. This is recommended, but the current usage is correct and there is no need to do all these modifications to be compliant with this rule. For the modifications: - SC2034: ksft_skip appears unused. - SC2154: optstring is referenced but not assigned. - SC2006: Use $(...) notation instead of legacy backticks `...`. Now this script is shellcheck (0.9.0) compliant. We can easily spot new issues. Reviewed-by:
-
Matthieu Baerts (NGI0) authored
shellcheck recently helped to prevent issues. It is then good to fix the other harmless issues in order to spot "real" ones later. Here, two categories of warnings are now ignored: - SC2317: Command appears to be unreachable. The cleanup() function is invoked indirectly via the EXIT trap. - SC2086: Double quote to prevent globbing and word splitting. This is recommended, but the current usage is correct and there is no need to do all these modifications to be compliant with this rule. For the modifications: - SC2034: ksft_skip appears unused. - SC2006: Use $(...) notation instead of legacy backticks `...`. - SC2145: Argument mixes string and array. Use * or separate argument. Now this script is shellcheck (0.9.0) compliant. We can easily spot new issues. Reviewed-by:
-
Matthieu Baerts (NGI0) authored
shellcheck recently helped to prevent issues. It is then good to fix the other harmless issues in order to spot "real" ones later. Here, two categories of warnings are now ignored: - SC2317: Command appears to be unreachable. The cleanup() function is invoked indirectly via the EXIT trap. - SC2086: Double quote to prevent globbing and word splitting. This is recommended, but the current usage is correct and there is no need to do all these modifications to be compliant with this rule. For the modifications: - SC2034: ksft_skip appears unused. - SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?. - SC2004: $/${} is unnecessary on arithmetic variables. - SC2155: Declare and assign separately to avoid masking return values. - SC2166: Prefer [ p ] && [ q ] as [ p -a q ] is not well defined. - SC2059: Don't use variables in the printf format string. Use printf '..%s..' "$foo". Now this script is shellcheck (0.9.0) compliant. We can easily spot new issues. Reviewed-by: -
Matthieu Baerts (NGI0) authored
shellcheck recently helped to prevent issues. It is then good to fix the other harmless issues in order to spot "real" ones later. Here, two categories of warnings are now ignored: - SC2317: Command appears to be unreachable. The cleanup() function is invoked indirectly via the EXIT trap. - SC2086: Double quote to prevent globbing and word splitting. This is recommended, but the current usage is correct and there is no need to do all these modifications to be compliant with this rule. For the modifications: - SC2034: ksft_skip appears unused. - SC2046: Quote '$(get_msk_inuse)' to prevent word splitting. - SC2006: Use $(...) notation instead of legacy backticks `...`. Now this script is shellcheck (0.9.0) compliant. We can easily spot new issues. Reviewed-by:
-
Geliang Tang authored
To avoid duplicated code in different MPTCP selftests, we can add and use helpers defined in mptcp_lib.sh. This patch unifies "pm_nl_ctl events" related code in userspace_pm.sh and mptcp_join.sh into a helper mptcp_lib_events(). Define it in mptcp_lib.sh and use it in both scripts. Note that mptcp_lib_kill_wait is now call before starting 'events' for mptcp_join.sh as well, but that's fine: each test is started from a new netns, so there will not be any existing pid there, and nothing is done when mptcp_lib_kill_wait is called with 0. Signed-off-by:
Geliang Tang <tanggeliang@kylinos.cn> Reviewed-by:
-
Geliang Tang authored
Set more the default sysctl values in mptcp_lib_ns_init(). It is fine to do that everywhere, because they could be overridden latter if needed. mptcp_lib_ns_exit() now also try to remove temp netns files used for the stats even for selftests not using them. That's fine to do that because these files have a unique name. Signed-off-by:
-
Geliang Tang authored
Add helpers mptcp_lib_ns_init() and mptcp_lib_ns_exit() in mptcp_lib.sh to initialize and delete the given namespaces. Then every test script can invoke these helpers and use all namespaces. Signed-off-by:
-
