• Magnus Karlsson's avatar
    xsk: add support for need_wakeup flag in AF_XDP rings · 77cd0d7b
    Magnus Karlsson authored
    This commit adds support for a new flag called need_wakeup in the
    AF_XDP Tx and fill rings. When this flag is set, it means that the
    application has to explicitly wake up the kernel Rx (for the bit in
    the fill ring) or kernel Tx (for bit in the Tx ring) processing by
    issuing a syscall. Poll() can wake up both depending on the flags
    submitted and sendto() will wake up tx processing only.
    
    The main reason for introducing this new flag is to be able to
    efficiently support the case when application and driver is executing
    on the same core. Previously, the driver was just busy-spinning on the
    fill ring if it ran out of buffers in the HW and there were none on
    the fill ring. This approach works when the application is running on
    another core as it can replenish the fill ring while the driver is
    busy-spinning. Though, this is a lousy approach if both of them are
    running on the same core as the probability of the fill ring getting
    more entries when the driver is busy-spinning is zero. With this new
    feature the driver now sets the need_wakeup flag and returns to the
    application. The application can then replenish the fill queue and
    then explicitly wake up the Rx processing in the kernel using the
    syscall poll(). For Tx, the flag is only set to one if the driver has
    no outstanding Tx completion interrupts. If it has some, the flag is
    zero as it will be woken up by a completion interrupt anyway.
    
    As a nice side effect, this new flag also improves the performance of
    the case where application and driver are running on two different
    cores as it reduces the number of syscalls to the kernel. The kernel
    tells user space if it needs to be woken up by a syscall, and this
    eliminates many of the syscalls.
    
    This flag needs some simple driver support. If the driver does not
    support this, the Rx flag is always zero and the Tx flag is always
    one. This makes any application relying on this feature default to the
    old behaviour of not requiring any syscalls in the Rx path and always
    having to call sendto() in the Tx path.
    
    For backwards compatibility reasons, this feature has to be explicitly
    turned on using a new bind flag (XDP_USE_NEED_WAKEUP). I recommend
    that you always turn it on as it so far always have had a positive
    performance impact.
    
    The name and inspiration of the flag has been taken from io_uring by
    Jens Axboe. Details about this feature in io_uring can be found in
    http://kernel.dk/io_uring.pdf, section 8.3.
    Signed-off-by: default avatarMagnus Karlsson <magnus.karlsson@intel.com>
    Acked-by: default avatarJonathan Lemon <jonathan.lemon@gmail.com>
    Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
    77cd0d7b
xdp_umem.c 8.77 KB