- 27 Jan, 2021 19 commits
-
-
Christoph Hellwig authored
bio_alloc never returns NULL when it can sleep. Signed-off-by:
Christoph Hellwig <hch@lst.de> Reviewed-by:
Johannes Thumshirn <johannes.thumshirn@wdc.com> Reviewed-by:
Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com> Acked-by:
Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by:
Jens Axboe <axboe@kernel.dk>
-
Christoph Hellwig authored
bio_alloc never returns NULL when it can sleep. Signed-off-by:
-
Christoph Hellwig authored
Refactor raid5_read_one_chunk so that all simple checks are done before allocating the bio. Signed-off-by:
Song Liu <song@kernel.org> Reviewed-by:
-
Christoph Hellwig authored
md_bio_alloc_sync is never called with a NULL mddev, and ->sync_set is initialized in md_run, so it always must be initialized as well. Just open code the remaining call to bio_alloc_bioset. Signed-off-by:
-
Christoph Hellwig authored
Use an on-stack bio and biovec for the single page synchronous I/O. Signed-off-by:
-
Christoph Hellwig authored
bio_alloc_mddev is never called with a NULL mddev, and ->bio_set is initialized in md_run, so it always must be initialized as well. Just open code the remaining call to bio_alloc_bioset. Signed-off-by:
-
Christoph Hellwig authored
Open code drbd_req_make_private_bio in the two callers to prepare for further changes. Also don't bother to initialize bi_next as the bio code already does that that. Signed-off-by:
-
Christoph Hellwig authored
Given that drbd_md_io_bio_set is initialized during module initialization and the module fails to load if the initialization fails there is no need to fall back to plain bio_alloc. Signed-off-by:
-
Christoph Hellwig authored
Sleeping bio allocations do not fail, which means that injecting an error into sleeping bio allocations is a little silly. Signed-off-by:
-
Christoph Hellwig authored
Use the blkdev_issue_flush helper instead of duplicating it. Signed-off-by:
-
Christoph Hellwig authored
Use blkdev_issue_flush instead of open coding it. Signed-off-by:
-
Christoph Hellwig authored
There is no point in allocating memory for a synchronous flush. Signed-off-by:
-
Christoph Hellwig authored
bio_kmalloc shares almost no logic with the bio_set based fast path in bio_alloc_bioset. Split it into an entirely separate implementation. Signed-off-by:
-
Christoph Hellwig authored
Use bio_kmalloc instead of open coding it. Signed-off-by:
Eric Biggers <ebiggers@google.com> Reviewed-by:
-
Christoph Hellwig authored
Use bio_kmalloc instead of open coding it. Signed-off-by:
Josef Bacik <josef@toxicpanda.com> Reviewed-by:
-
Christoph Hellwig authored
Use bio_alloc instead of open coding it. Signed-off-by:
-
Jan Kara authored
Currently whenever bfq queue has a request queued we add now - last_completion_time to the think time statistics. This is however misleading in case the process is able to submit several requests in parallel because e.g. if the queue has request completed at time T0 and then queues new requests at times T1, T2, then we will add T1-T0 and T2-T0 to think time statistics which just doesn't make any sence (the queue's think time is penalized by the queue being able to submit more IO). So add to think time statistics only time intervals when the queue had no IO pending. Signed-off-by:
Jan Kara <jack@suse.cz> Acked-by:
Paolo Valente <paolo.valente@linaro.org> [axboe: fix whitespace on empty line] Signed-off-by:
-
Jan Kara authored
Use local variable 'ttime' instead of dereferencing bfqq. Signed-off-by:
-
Jan Kara authored
bfq_setup_cooperator() uses bfqd->in_serv_last_pos so detect whether it makes sense to merge current bfq queue with the in-service queue. However if the in-service queue is freshly scheduled and didn't dispatch any requests yet, bfqd->in_serv_last_pos is stale and contains value from the previously scheduled bfq queue which can thus result in a bogus decision that the two queues should be merged. This bug can be observed for example with the following fio jobfile: [global] direct=0 ioengine=sync invalidate=1 size=1g rw=read [reader] numjobs=4 directory=/mnt where the 4 processes will end up in the one shared bfq queue although they do IO to physically very distant files (for some reason I was able to observe this only with slice_idle=1ms setting). Fix the problem by invalidating bfqd->in_serv_last_pos when switching in-service queue. Fixes: 058fdecc ("block, bfq: fix in-service-queue check for queue merging") CC: stable@vger.kernel.org Signed-off-by:
-
- 26 Jan, 2021 8 commits
-
-
Chunguang Xu authored
When calling blkcg_schedule_throttle(), for the same queue, redundant get/put operations can be removed. Signed-off-by:
Chunguang Xu <brookxu@tencent.com> Acked-by:
Tejun Heo <tj@kernel.org> Signed-off-by:
-
Christoph Hellwig authored
truncate_bdev_range is only used in always built-in block layer code, so remove the export and the !CONFIG_BLOCK stub. Signed-off-by:
-
Lei Chen authored
The first parameter rwb is not used for this function. So just remove it. Signed-off-by:
Lei Chen <lennychen@tencent.com> Signed-off-by:
-
Jan Kara authored
blkdev_fallocate() tries to detect whether a discard raced with an overlapping write by calling invalidate_inode_pages2_range(). However this check can give both false negatives (when writing using direct IO or when writeback already writes out the written pagecache range) and false positives (when write is not actually overlapping but ends in the same page when blocksize < pagesize). This actually causes issues for qemu which is getting confused by EBUSY errors. Fix the problem by removing this conflicting write detection since it is inherently racy and thus of little use anyway. Reported-by:
Maxim Levitsky <mlevitsk@redhat.com> CC: "Darrick J. Wong" <darrick.wong@oracle.com> Link: https://lore.kernel.org/qemu-devel/20201111153913.41840-1-mlevitsk@redhat.comSigned-off-by:
Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by:
-
Christoph Hellwig authored
Cloned bios are can be used to on the same device, in which case we need to inherit the BIO_REMAPPED flag to avoid a double partition remap. When the cloned bios are used on another device, bio_set_dev will clear the flag. Fixes: 309dca30 ("block: store a block_device pointer in struct bio") Signed-off-by:
-
Christoph Hellwig authored
Always use the bio_set_dev helper to assign ->bi_bdev to make sure other state related to the device is uptodate. Signed-off-by:
-
Christoph Hellwig authored
Always use the bio_set_dev helper to assign ->bi_bdev to make sure other state related to the device is uptodate. Signed-off-by:
-
Jens Axboe authored
It's only used in the same file, mark is appropriately static. Fixes: 71217df3 ("block, bfq: make waker-queue detection more robust") Reported-by:
kernel test robot <lkp@intel.com> Signed-off-by:
-
- 25 Jan, 2021 13 commits
-
-
Paolo Valente authored
In the presence of many parallel I/O flows, the detection of waker bfq_queues suffers from false positives. This commits addresses this issue by making the filtering of actual wakers more selective. In more detail, a candidate waker must be found to meet waker requirements three times before being promoted to actual waker. Tested-by:
-
Paolo Valente authored
To prevent injection information from being lost on bfq_queue merging, also the amount of service that a bfq_queue receives must be saved and restored when the bfq_queue is merged and split, respectively. Tested-by:
-
Paolo Valente authored
To prevent weight-raising information from being lost on bfq_queue merging, also the amount of service that a bfq_queue receives must be saved and restored when the bfq_queue is merged and split, respectively. Tested-by:
-
Paolo Valente authored
A bfq_queue may happen to be deemed as soft real-time while it is still enjoying interactive weight-raising. If this happens because of a false positive, then the bfq_queue is likely to loose its soft real-time status soon. Upon losing such a status, the bfq_queue must get back its interactive weight-raising, if its interactive period is not over yet. But this case is not handled. This commit corrects this error. Tested-by:
-
Paolo Valente authored
Upon an I/O-dispatch attempt, BFQ may detect that it was better to plug I/O dispatch, and to wait for a new request to arrive for the currently in-service queue. But the arrival of a new request for an empty bfq_queue, and thus the switch from idle to busy of the bfq_queue, may cause the scenario to change, and make plugging no longer needed for service guarantees, or more convenient for throughput. In this case, keeping I/O-dispatch plugged would certainly lower throughput. To address this issue, this commit makes such a check, and stops plugging I/O if it is better to stop plugging I/O. Tested-by:
-
Paolo Valente authored
Some BFQ mechanisms make their decisions on a bfq_queue basing also on whether the bfq_queue is I/O bound. In this respect, the current logic for evaluating whether a bfq_queue is I/O bound is rather rough. This commits replaces this logic with a more effective one. The new logic measures the percentage of time during which a bfq_queue is active, and marks the bfq_queue as I/O bound if the latter if this percentage is above a fixed threshold. Tested-by:
-
Christoph Hellwig authored
When an already remapped bio is resubmitted (e.g. by blk_queue_split), bio_check_eod will compare the remapped bi_sector against the size of the partition, leading to spurious I/O failures. Skip the EOD check in this case. Fixes: 309dca30 ("block: store a block_device pointer in struct bio") Reported-by:
-
Pavel Begunkov authored
The block layer spends quite a while in blkdev_direct_IO() to copy and initialise bio's bvec. However, if we've already got a bvec in the input iterator it might be reused in some cases, i.e. when new ITER_BVEC_FLAG_FIXED flag is set. Simple tests show considerable performance boost, and it also reduces memory footprint. Suggested-by:
Matthew Wilcox <willy@infradead.org> Reviewed-by:
Pavel Begunkov <asml.silence@gmail.com> Reviewed-by:
Ming Lei <ming.lei@redhat.com> Signed-off-by:
-
Pavel Begunkov authored
Add a helper function calculating the number of bvec segments we need to allocate to construct a bio. It doesn't change anything functionally, but will be used to not duplicate special cases in the future. Reviewed-by:
-
Pavel Begunkov authored
iov_iter_advance() is heavily used, but implemented through generic means. For bvecs there is a specifically crafted function for that, so use bvec_iter_advance() instead, it's faster and slimmer. Reviewed-by:
-
Christoph Hellwig authored
This saves one memory allocation, and ensures the bvecs aren't freed before the AIO completion. This will allow the lower level code to be optimized so that it can avoid allocating another bvec array. Signed-off-by:
-
Pavel Begunkov authored
Direct IO does not operate on the current working set of pages managed by the kernel, so it should not be accounted as memory stall to PSI infrastructure. The block layer and iomap direct IO use bio_iov_iter_get_pages() to build bios, and they are the only users of it, so to avoid PSI tracking for them clear out BIO_WORKINGSET flag. Do same for dio_bio_submit() because fs/direct_io constructs bios by hand directly calling bio_add_page(). Reported-by:
Christoph Hellwig <hch@infradead.org> Suggested-by:
Johannes Weiner <hannes@cmpxchg.org> Reviewed-by:
-
Pavel Begunkov authored
zero-length bvec segments are allowed in general, but not handled by bio and down the block layer so filtered out. This inconsistency may be confusing and prevent from optimisations. As zero-length segments are useless and places that were generating them are patched, declare them not allowed. Reviewed-by:
-
