• Richard Gobert's avatar
    net: gro: move L3 flush checks to tcp_gro_receive and udp_gro_receive_segment · 4b0ebbca
    Richard Gobert authored
    {inet,ipv6}_gro_receive functions perform flush checks (ttl, flags,
    iph->id, ...) against all packets in a loop. These flush checks are used in
    all merging UDP and TCP flows.
    
    These checks need to be done only once and only against the found p skb,
    since they only affect flush and not same_flow.
    
    This patch leverages correct network header offsets from the cb for both
    outer and inner network headers - allowing these checks to be done only
    once, in tcp_gro_receive and udp_gro_receive_segment. As a result,
    NAPI_GRO_CB(p)->flush is not used at all. In addition, flush_id checks are
    more declarative and contained in inet_gro_flush, thus removing the need
    for flush_id in napi_gro_cb.
    
    This results in less parsing code for non-loop flush tests for TCP and UDP
    flows.
    
    To make sure results are not within noise range - I've made netfilter drop
    all TCP packets, and measured CPU performance in GRO (in this case GRO is
    responsible for about 50% of the CPU utilization).
    
    perf top while replaying 64 parallel IP/TCP streams merging in GRO:
    (gro_receive_network_flush is compiled inline to tcp_gro_receive)
    net-next:
            6.94% [kernel] [k] inet_gro_receive
            3.02% [kernel] [k] tcp_gro_receive
    
    patch applied:
            4.27% [kernel] [k] tcp_gro_receive
            4.22% [kernel] [k] inet_gro_receive
    
    perf top while replaying 64 parallel IP/IP/TCP streams merging in GRO (same
    results for any encapsulation, in this case inet_gro_receive is top
    offender in net-next)
    net-next:
            10.09% [kernel] [k] inet_gro_receive
            2.08% [kernel] [k] tcp_gro_receive
    
    patch applied:
            6.97% [kernel] [k] inet_gro_receive
            3.68% [kernel] [k] tcp_gro_receive
    Signed-off-by: default avatarRichard Gobert <richardbgobert@gmail.com>
    Reviewed-by: default avatarWillem de Bruijn <willemb@google.com>
    Link: https://lore.kernel.org/r/20240509190819.2985-3-richardbgobert@gmail.comSigned-off-by: default avatarJakub Kicinski <kuba@kernel.org>
    4b0ebbca
gro.h 15.4 KB