Commit 4140aafc authored by Olivier Bacon's avatar Olivier Bacon Committed by Herbert Xu

crypto: engine - fix crypto_queue backlog handling

CRYPTO_TFM_REQ_MAY_BACKLOG tells the crypto driver that it should
internally backlog requests until the crypto hw's queue becomes
full. At that point, crypto_engine backlogs the request and returns
-EBUSY. Calling driver such as dm-crypt then waits until the
complete() function is called with a status of -EINPROGRESS before
sending a new request.

The problem lies in the call to complete() with a value of -EINPROGRESS
that is made when a backlog item is present on the queue. The call is
done before the successful execution of the crypto request. In the case
that do_one_request() returns < 0 and the retry support is available,
the request is put back in the queue. This leads upper drivers to send
a new request even if the queue is still full.

The problem can be reproduced by doing a large dd into a crypto
dm-crypt device. This is pretty easy to see when using
Freescale CAAM crypto driver and SWIOTLB dma. Since the actual amount
of requests that can be hold in the queue is unlimited we get IOs error
and dma allocation.

The fix is to call complete with a value of -EINPROGRESS only if
the request is not enqueued back in crypto_queue. This is done
by calling complete() later in the code. In order to delay the decision,
crypto_queue is modified to correctly set the backlog pointer
when a request is enqueued back.

Fixes: 6a89f492 ("crypto: engine - support for parallel requests based on retry mechanism")
Co-developed-by: default avatarSylvain Ouellet <souellet@genetec.com>
Signed-off-by: default avatarSylvain Ouellet <souellet@genetec.com>
Signed-off-by: default avatarOlivier Bacon <obacon@genetec.com>
Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
parent 8fd91151
...@@ -961,6 +961,9 @@ EXPORT_SYMBOL_GPL(crypto_enqueue_request); ...@@ -961,6 +961,9 @@ EXPORT_SYMBOL_GPL(crypto_enqueue_request);
void crypto_enqueue_request_head(struct crypto_queue *queue, void crypto_enqueue_request_head(struct crypto_queue *queue,
struct crypto_async_request *request) struct crypto_async_request *request)
{ {
if (unlikely(queue->qlen >= queue->max_qlen))
queue->backlog = queue->backlog->prev;
queue->qlen++; queue->qlen++;
list_add(&request->list, &queue->list); list_add(&request->list, &queue->list);
} }
......
...@@ -129,9 +129,6 @@ static void crypto_pump_requests(struct crypto_engine *engine, ...@@ -129,9 +129,6 @@ static void crypto_pump_requests(struct crypto_engine *engine,
if (!engine->retry_support) if (!engine->retry_support)
engine->cur_req = async_req; engine->cur_req = async_req;
if (backlog)
crypto_request_complete(backlog, -EINPROGRESS);
if (engine->busy) if (engine->busy)
was_busy = true; was_busy = true;
else else
...@@ -217,6 +214,9 @@ static void crypto_pump_requests(struct crypto_engine *engine, ...@@ -217,6 +214,9 @@ static void crypto_pump_requests(struct crypto_engine *engine,
crypto_request_complete(async_req, ret); crypto_request_complete(async_req, ret);
retry: retry:
if (backlog)
crypto_request_complete(backlog, -EINPROGRESS);
/* If retry mechanism is supported, send new requests to engine */ /* If retry mechanism is supported, send new requests to engine */
if (engine->retry_support) { if (engine->retry_support) {
spin_lock_irqsave(&engine->queue_lock, flags); spin_lock_irqsave(&engine->queue_lock, flags);
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment