• Toshiaki Makita's avatar
    veth: Support bulk XDP_TX · 9cda7807
    Toshiaki Makita authored
    XDP_TX is similar to XDP_REDIRECT as it essentially redirects packets to
    the device itself. XDP_REDIRECT has bulk transmit mechanism to avoid the
    heavy cost of indirect call but it also reduces lock acquisition on the
    destination device that needs locks like veth and tun.
    
    XDP_TX does not use indirect calls but drivers which require locks can
    benefit from the bulk transmit for XDP_TX as well.
    
    This patch introduces bulk transmit mechanism in veth using bulk queue
    on stack, and improves XDP_TX performance by about 9%.
    
    Here are single-core/single-flow XDP_TX test results. CPU consumptions
    are taken from "perf report --no-child".
    
    - Before:
    
      7.26 Mpps
    
      _raw_spin_lock  7.83%
      veth_xdp_xmit  12.23%
    
    - After:
    
      7.94 Mpps
    
      _raw_spin_lock  1.08%
      veth_xdp_xmit   6.10%
    
    v2:
    - Use stack for bulk queue instead of a global variable.
    Signed-off-by: default avatarToshiaki Makita <toshiaki.makita1@gmail.com>
    Acked-by: default avatarJesper Dangaard Brouer <brouer@redhat.com>
    Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
    9cda7807
veth.c 31.6 KB