1. 17 Nov, 2010 39 commits
  2. 16 Nov, 2010 1 commit
    • Eric Dumazet's avatar
      net: reorder struct sock fields · b178bb3d
      Eric Dumazet authored
      Right now, fields in struct sock are not optimally ordered, because each
      path (RX softirq, TX completion, RX user,  TX user) has to touch fields
      that are contained in many different cache lines.
      
      The really critical thing is to shrink number of cache lines that are
      used at RX softirq time : CPU handling softirqs for a device can receive
      many frames per second for many sockets. If load is too big, we can drop
      frames at NIC level. RPS or multiqueue cards can help, but better reduce
      latency if possible.
      
      This patch starts with UDP protocol, then additional patches will try to
      reduce latencies of other ones as well.
      
      At RX softirq time, fields of interest for UDP protocol are :
      (not counting ones in inet struct for the lookup)
      
      Read/Written:
      sk_refcnt   (atomic increment/decrement)
      sk_rmem_alloc & sk_backlog.len (to check if there is room in queues)
      sk_receive_queue
      sk_backlog (if socket locked by user program)
      sk_rxhash
      sk_forward_alloc
      sk_drops
      
      Read only:
      sk_rcvbuf (sk_rcvqueues_full())
      sk_filter
      sk_wq
      sk_policy[0]
      sk_flags
      
      Additional notes :
      
      - sk_backlog has one hole on 64bit arches. We can fill it to save 8
      bytes.
      - sk_backlog is used only if RX sofirq handler finds the socket while
      locked by user.
      - sk_rxhash is written only once per flow.
      - sk_drops is written only if queues are full
      
      Final layout :
      
      [1] One section grouping all read/write fields, but placing rxhash and
      sk_backlog at the end of this section.
      
      [2] One section grouping all read fields in RX handler
         (sk_filter, sk_rcv_buf, sk_wq)
      
      [3] Section used by other paths
      
      I'll post a patch on its own to put sk_refcnt at the end of struct
      sock_common so that it shares same cache line than section [1]
      
      New offsets on 64bit arch :
      
      sizeof(struct sock)=0x268
      offsetof(struct sock, sk_refcnt)  =0x10
      offsetof(struct sock, sk_lock)    =0x48
      offsetof(struct sock, sk_receive_queue)=0x68
      offsetof(struct sock, sk_backlog)=0x80
      offsetof(struct sock, sk_rmem_alloc)=0x80
      offsetof(struct sock, sk_forward_alloc)=0x98
      offsetof(struct sock, sk_rxhash)=0x9c
      offsetof(struct sock, sk_rcvbuf)=0xa4
      offsetof(struct sock, sk_drops) =0xa0
      offsetof(struct sock, sk_filter)=0xa8
      offsetof(struct sock, sk_wq)=0xb0
      offsetof(struct sock, sk_policy)=0xd0
      offsetof(struct sock, sk_flags) =0xe0
      
      Instead of :
      
      sizeof(struct sock)=0x270
      offsetof(struct sock, sk_refcnt)  =0x10
      offsetof(struct sock, sk_lock)    =0x50
      offsetof(struct sock, sk_receive_queue)=0xc0
      offsetof(struct sock, sk_backlog)=0x70
      offsetof(struct sock, sk_rmem_alloc)=0xac
      offsetof(struct sock, sk_forward_alloc)=0x10c
      offsetof(struct sock, sk_rxhash)=0x128
      offsetof(struct sock, sk_rcvbuf)=0x4c
      offsetof(struct sock, sk_drops) =0x16c
      offsetof(struct sock, sk_filter)=0x198
      offsetof(struct sock, sk_wq)=0x88
      offsetof(struct sock, sk_policy)=0x98
      offsetof(struct sock, sk_flags) =0x130
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      b178bb3d