• Usama Arif's avatar
    io_uring: avoid ring quiesce while registering/unregistering eventfd · 77bc59b4
    Usama Arif authored
    This is done by creating a new RCU data structure (io_ev_fd) as part of
    io_ring_ctx that holds the eventfd_ctx.
    
    The function io_eventfd_signal is executed under rcu_read_lock with a
    single rcu_dereference to io_ev_fd so that if another thread unregisters
    the eventfd while io_eventfd_signal is still being executed, the
    eventfd_signal for which io_eventfd_signal was called completes
    successfully.
    
    The process of registering/unregistering eventfd is already done under
    uring_lock so multiple threads won't enter a race condition while
    registering/unregistering eventfd.
    
    With the above approach ring quiesce can be avoided which is much more
    expensive then using RCU lock. On the system tested, io_uring_register
    with IORING_REGISTER_EVENTFD takes less than 1ms with RCU lock, compared
    to 15ms before with ring quiesce.
    Signed-off-by: default avatarUsama Arif <usama.arif@bytedance.com>
    Link: https://lore.kernel.org/r/20220204145117.1186568-3-usama.arif@bytedance.com
    [axboe: long line fixups]
    Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
    77bc59b4
io_uring.c 279 KB