• Jens Axboe's avatar
    io_uring: ensure that cached task references are always put on exit · e775f93f
    Jens Axboe authored
    io_uring caches task references to avoid doing atomics for each of them
    per request. If a request is put from the same task that allocated it,
    then we can maintain a per-ctx cache of them. This obviously relies
    on io_uring always pruning caches in a reliable way, and there's
    currently a case off io_uring fd release where we can miss that.
    
    One example is a ring setup with IOPOLL, which relies on the task
    polling for completions, which will free them. However, if such a task
    submits a request and then exits or closes the ring without reaping
    the completion, then ring release will reap and put. If release happens
    from that very same task, the completed request task refs will get
    put back into the cache pool. This is problematic, as we're now beyond
    the point of pruning caches.
    
    Manually drop these caches after doing an IOPOLL reap. This releases
    references from the current task, which is enough. If another task
    happens to be doing the release, then the caching will not be
    triggered and there's no issue.
    
    Cc: stable@vger.kernel.org
    Fixes: e98e49b2 ("io_uring: extend task put optimisations")
    Reported-by: default avatarHomin Rhee <hominlab@gmail.com>
    Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
    e775f93f
io_uring.c 101 KB