• Jesper Dangaard Brouer's avatar
    qdisc: bulk dequeue support for qdiscs with TCQ_F_ONETXQUEUE · 5772e9a3
    Jesper Dangaard Brouer authored
    Based on DaveM's recent API work on dev_hard_start_xmit(), that allows
    sending/processing an entire skb list.
    
    This patch implements qdisc bulk dequeue, by allowing multiple packets
    to be dequeued in dequeue_skb().
    
    The optimization principle for this is two fold, (1) to amortize
    locking cost and (2) avoid expensive tailptr update for notifying HW.
     (1) Several packets are dequeued while holding the qdisc root_lock,
    amortizing locking cost over several packet.  The dequeued SKB list is
    processed under the TXQ lock in dev_hard_start_xmit(), thus also
    amortizing the cost of the TXQ lock.
     (2) Further more, dev_hard_start_xmit() will utilize the skb->xmit_more
    API to delay HW tailptr update, which also reduces the cost per
    packet.
    
    One restriction of the new API is that every SKB must belong to the
    same TXQ.  This patch takes the easy way out, by restricting bulk
    dequeue to qdisc's with the TCQ_F_ONETXQUEUE flag, that specifies the
    qdisc only have attached a single TXQ.
    
    Some detail about the flow; dev_hard_start_xmit() will process the skb
    list, and transmit packets individually towards the driver (see
    xmit_one()).  In case the driver stops midway in the list, the
    remaining skb list is returned by dev_hard_start_xmit().  In
    sch_direct_xmit() this returned list is requeued by dev_requeue_skb().
    
    To avoid overshooting the HW limits, which results in requeuing, the
    patch limits the amount of bytes dequeued, based on the drivers BQL
    limits.  In-effect bulking will only happen for BQL enabled drivers.
    
    Small amounts for extra HoL blocking (2x MTU/0.24ms) were
    measured at 100Mbit/s, with bulking 8 packets, but the
    oscillating nature of the measurement indicate something, like
    sched latency might be causing this effect. More comparisons
    show, that this oscillation goes away occationally. Thus, we
    disregard this artifact completely and remove any "magic" bulking
    limit.
    
    For now, as a conservative approach, stop bulking when seeing TSO and
    segmented GSO packets.  They already benefit from bulking on their own.
    A followup patch add this, to allow easier bisect-ability for finding
    regressions.
    
    Jointed work with Hannes, Daniel and Florian.
    Signed-off-by: default avatarJesper Dangaard Brouer <brouer@redhat.com>
    Signed-off-by: default avatarHannes Frederic Sowa <hannes@stressinduktion.org>
    Signed-off-by: default avatarDaniel Borkmann <dborkman@redhat.com>
    Signed-off-by: default avatarFlorian Westphal <fw@strlen.de>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    5772e9a3
sch_generic.h 19.6 KB