Commit f959325e authored by Nathan Huckleberry's avatar Nathan Huckleberry Committed by Eric Biggers

fsverity: Remove WQ_UNBOUND from fsverity read workqueue

WQ_UNBOUND causes significant scheduler latency on ARM64/Android.  This
is problematic for latency sensitive workloads, like I/O
post-processing.

Removing WQ_UNBOUND gives a 96% reduction in fsverity workqueue related
scheduler latency and improves app cold startup times by ~30ms.
WQ_UNBOUND was also removed from the dm-verity workqueue for the same
reason [1].

This code was tested by running Android app startup benchmarks and
measuring how long the fsverity workqueue spent in the runnable state.

Before
Total workqueue scheduler latency: 553800us
After
Total workqueue scheduler latency: 18962us

[1]: https://lore.kernel.org/all/20230202012348.885402-1-nhuck@google.com/Signed-off-by: default avatarNathan Huckleberry <nhuck@google.com>
Fixes: 8a1d0f9c ("fs-verity: add data verification hooks for ->readpages()")
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/r/20230310193325.620493-1-nhuck@google.comSigned-off-by: default avatarEric Biggers <ebiggers@google.com>
parent eeac8ede
...@@ -387,15 +387,15 @@ EXPORT_SYMBOL_GPL(fsverity_enqueue_verify_work); ...@@ -387,15 +387,15 @@ EXPORT_SYMBOL_GPL(fsverity_enqueue_verify_work);
int __init fsverity_init_workqueue(void) int __init fsverity_init_workqueue(void)
{ {
/* /*
* Use an unbound workqueue to allow bios to be verified in parallel * Use a high-priority workqueue to prioritize verification work, which
* even when they happen to complete on the same CPU. This sacrifices * blocks reads from completing, over regular application tasks.
* locality, but it's worthwhile since hashing is CPU-intensive.
* *
* Also use a high-priority workqueue to prioritize verification work, * For performance reasons, don't use an unbound workqueue. Using an
* which blocks reads from completing, over regular application tasks. * unbound workqueue for crypto operations causes excessive scheduler
* latency on ARM64.
*/ */
fsverity_read_workqueue = alloc_workqueue("fsverity_read_queue", fsverity_read_workqueue = alloc_workqueue("fsverity_read_queue",
WQ_UNBOUND | WQ_HIGHPRI, WQ_HIGHPRI,
num_online_cpus()); num_online_cpus());
if (!fsverity_read_workqueue) if (!fsverity_read_workqueue)
return -ENOMEM; return -ENOMEM;
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment