• Pavel Begunkov's avatar
    io_uring: fix racy req->flags modification · bb175342
    Pavel Begunkov authored
    Setting and clearing REQ_F_OVERFLOW in io_uring_cancel_files() and
    io_cqring_overflow_flush() are racy, because they might be called
    asynchronously.
    
    REQ_F_OVERFLOW flag in only needed for files cancellation, so if it can
    be guaranteed that requests _currently_ marked inflight can't be
    overflown, the problem will be solved with removing the flag
    altogether.
    
    That's how the patch works, it removes inflight status of a request
    in io_cqring_fill_event() whenever it should be thrown into CQ-overflow
    list. That's Ok to do, because no opcode specific handling can be done
    after io_cqring_fill_event(), the same assumption as with "struct
    io_completion" patches.
    And it already have a good place for such cleanups, which is
    io_clean_op(). A nice side effect of this is removing this inflight
    check from the hot path.
    
    note on synchronisation: now __io_cqring_fill_event() may be taking two
    spinlocks simultaneously, completion_lock and inflight_lock. It's fine,
    because we never do that in reverse order, and CQ-overflow of inflight
    requests shouldn't happen often.
    Signed-off-by: default avatarPavel Begunkov <asml.silence@gmail.com>
    Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
    bb175342
io_uring.c 210 KB