• Jens Axboe's avatar
    iomap: support IOCB_DIO_CALLER_COMP · 8c052fb3
    Jens Axboe authored
    If IOCB_DIO_CALLER_COMP is set, utilize that to set kiocb->dio_complete
    handler and data for that callback. Rather than punt the completion to a
    workqueue, we pass back the handler and data to the issuer and will get
    a callback from a safe task context.
    
    Using the following fio job to randomly dio write 4k blocks at
    queue depths of 1..16:
    
    fio --name=dio-write --filename=/data1/file --time_based=1 \
    --runtime=10 --bs=4096 --rw=randwrite --norandommap --buffered=0 \
    --cpus_allowed=4 --ioengine=io_uring --iodepth=$depth
    
    shows the following results before and after this patch:
    
    	Stock	Patched		Diff
    =======================================
    QD1	155K	162K		+ 4.5%
    QD2	290K	313K		+ 7.9%
    QD4	533K	597K		+12.0%
    QD8	604K	827K		+36.9%
    QD16	615K	845K		+37.4%
    
    which shows nice wins all around. If we factored in per-IOP efficiency,
    the wins look even nicer. This becomes apparent as queue depth rises,
    as the offloaded workqueue completions runs out of steam.
    Reviewed-by: default avatarDarrick J. Wong <djwong@kernel.org>
    Reviewed-by: default avatarChristoph Hellwig <hch@lst.de>
    Reviewed-by: default avatarDave Chinner <dchinner@redhat.com>
    Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
    8c052fb3
direct-io.c 21.4 KB