• Eric Dumazet's avatar
    net_sched: sch_fq: add fast path for mostly idle qdisc · 076433bd
    Eric Dumazet authored
    TCQ_F_CAN_BYPASS can be used by few qdiscs.
    
    Idea is that if we queue a packet to an empty qdisc,
    following dequeue() would pick it immediately.
    
    FQ can not use the generic TCQ_F_CAN_BYPASS code,
    because some additional checks need to be performed.
    
    This patch adds a similar fast path to FQ.
    
    Most of the time, qdisc is not throttled,
    and many packets can avoid bringing/touching
    at least four cache lines, and consuming 128bytes
    of memory to store the state of a flow.
    
    After this patch, netperf can send UDP packets about 13 % faster,
    and pktgen goes 30 % faster (when FQ is in the way), on a fast NIC.
    
    TCP traffic is also improved, thanks to a reduction of cache line misses.
    I have measured a 5 % increase of throughput on a tcp_rr intensive workload.
    
    tc -s -d qd sh dev eth1
    ...
    qdisc fq 8004: parent 1:2 limit 10000p flow_limit 100p buckets 1024
       orphan_mask 1023 quantum 3028b initial_quantum 15140b low_rate_threshold 550Kbit
       refill_delay 40ms timer_slack 10us horizon 10s horizon_drop
     Sent 5646784384 bytes 1985161 pkt (dropped 0, overlimits 0 requeues 0)
     backlog 0b 0p requeues 0
      flows 122 (inactive 122 throttled 0)
      gc 0 highprio 0 fastpath 659990 throttled 27762 latency 8.57us
    Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    076433bd
sch_fq.c 28.2 KB