• Willem de Bruijn's avatar
    tcp: enable MSG_ZEROCOPY · f214f915
    Willem de Bruijn authored
    Enable support for MSG_ZEROCOPY to the TCP stack. TSO and GSO are
    both supported. Only data sent to remote destinations is sent without
    copying. Packets looped onto a local destination have their payload
    copied to avoid unbounded latency.
    
    Tested:
      A 10x TCP_STREAM between two hosts showed a reduction in netserver
      process cycles by up to 70%, depending on packet size. Systemwide,
      savings are of course much less pronounced, at up to 20% best case.
    
      msg_zerocopy.sh 4 tcp:
    
      without zerocopy
        tx=121792 (7600 MB) txc=0 zc=n
        rx=60458 (7600 MB)
    
      with zerocopy
        tx=286257 (17863 MB) txc=286257 zc=y
        rx=140022 (17863 MB)
    
      This test opens a pair of sockets over veth, one one calls send with
      64KB and optionally MSG_ZEROCOPY and on the other reads the initial
      bytes. The receiver truncates, so this is strictly an upper bound on
      what is achievable. It is more representative of sending data out of
      a physical NIC (when payload is not touched, either).
    Signed-off-by: default avatarWillem de Bruijn <willemb@google.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    f214f915
tcp.c 89.4 KB