The req_lock is currently implemented as a rw_lock, but there are no
instances where read_lock() is called. This means that the lock is
effectively only used by writers, making it functionally equivalent to
a simple spinlock.

As stated in Documentation/locking/spinlocks.rst:
"Reader-writer locks require more atomic memory operations than simple
spinlocks. Unless the reader critical section is long, you are better
off just using spinlocks."

Since the rw_lock in this case incurs additional atomic memory
operations without any benefit from reader-writer locking, it is more
efficient to replace it with a spinlock. This patch implements that
replacement to optimize the driver's performance.
Signed-off-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Simplify the code by replacing devm_clk_get() and clk_prepare_enable()
with devm_clk_get_enabled(), which also avoids the call to
clk_disable_unprepare().
Signed-off-by: Chunhai Guo <guochunhai@vivo.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Simplify the code by replacing devm_clk_get() and clk_prepare() with
devm_clk_get_prepared(), which also avoids the call to clk_unprepare().
Signed-off-by: Chunhai Guo <guochunhai@vivo.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

In the case where we are forcing the ps.chunk_size to be at least 1,
we are ignoring the caller's alignment.

Move the forcing of ps.chunk_size to be at least 1 before rounding it
up to caller's alignment, so that caller's alignment is honored.

While at it, use max() to force the ps.chunk_size to be at least 1 to
improve readability.

Fixes: 6d45e1c9 ("padata: Fix possible divide-by-0 panic in padata_mt_helper()")
Signed-off-by: Kamlesh Gurudasani <kamlesh@ti.com>
Acked-by:   Waiman Long <longman@redhat.com>
Acked-by: Daniel Jordan <daniel.m.jordan@oracle.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Add two description for register space of rtic. There are two register
space, one is for control and status, the other optional space is
recoverable error indication register space.

Fix below CHECK_DTBS error:
arch/arm64/boot/dts/freescale/fsl-ls1012a-frdm.dtb: crypto@1700000: rtic@60000:reg: [[393216, 256], [396800, 24]] is too long
        from schema $id: http://devicetree.org/schemas/crypto/fsl,sec-v4.0.yaml#Signed-off-by: Frank Li <Frank.Li@nxp.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

It's unlikely that devm_pm_runtime_enable ever fails. Still, it makes
sense to read the return value and handle errors.
Signed-off-by: Martin Kaiser <martin@kaiser.cx>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The driver uses the rst variable only for an initial reset when the chip
is probed. There's no need to store rst in the driver's private data, we
can make it a local variable in the probe function.
Signed-off-by: Martin Kaiser <martin@kaiser.cx>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The devm_clk_get_enabled() helpers:
    - call devm_clk_get()
    - call clk_prepare_enable() and register what is needed in order to
     call clk_disable_unprepare() when needed, as a managed resource.

This simplifies the code and avoids the calls to clk_disable_unprepare().
Signed-off-by: Huan Yang <link@vivo.com>
Reviewed-by: Martin Kaiser <martin@kaiser.cx>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

This function is never implemented and used since introduction in
commit 049359d6 ("crypto: amcc - Add crypt4xx driver").
Signed-off-by: Yue Haibing <yuehaibing@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

This function is never implemented and used since introduction in
commit 48fe583f ("crypto: amlogic - Add crypto accelerator for
amlogic GXL").
Signed-off-by: Yue Haibing <yuehaibing@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

This function is never implemented and used since introduction in
commit 720419f0 ("crypto: ccp - Introduce the AMD Secure Processor
device").
Signed-off-by: Yue Haibing <yuehaibing@huawei.com>
Acked-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

This function is never implemented and used since introduction in
commit 10b4f094 ("crypto: marvell - add the Virtual Function
driver for CPT")
Signed-off-by: Yue Haibing <yuehaibing@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

This function is never implemented and used since introduction in
commit 46c5338d ("crypto: sl3516 - Add sl3516 crypto engine")
Signed-off-by: Yue Haibing <yuehaibing@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Commit 9744fec9 ("crypto: inside-secure - remove request list to
improve performance") declar this but never implemented.
Signed-off-by: Yue Haibing <yuehaibing@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

the variable is never referenced in the code, just remove them.
Signed-off-by: Zhu Jun <zhujun2@cmss.chinamobile.com>
Reviewed-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Use the min() macro to simplify the jent_read_entropy() function and
improve its readability.
Signed-off-by: Thorsten Blum <thorsten.blum@toblux.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

In case of sev PLATFORM_STATUS failure, sev_get_api_version() fails
resulting in sev_data field of psp_master nulled out. This later becomes
a problem when unloading the ccp module because the device has not been
unregistered (via misc_deregister()) before clearing the sev_data field
of psp_master. As a result, on reloading the ccp module, a duplicate
device issue is encountered as can be seen from the dmesg log below.

on reloading ccp module via modprobe ccp

Call Trace:
  <TASK>
  dump_stack_lvl+0xd7/0xf0
  dump_stack+0x10/0x20
  sysfs_warn_dup+0x5c/0x70
  sysfs_create_dir_ns+0xbc/0xd
  kobject_add_internal+0xb1/0x2f0
  kobject_add+0x7a/0xe0
  ? srso_alias_return_thunk+0x5/0xfbef5
  ? get_device_parent+0xd4/0x1e0
  ? __pfx_klist_children_get+0x10/0x10
  device_add+0x121/0x870
  ? srso_alias_return_thunk+0x5/0xfbef5
  device_create_groups_vargs+0xdc/0x100
  device_create_with_groups+0x3f/0x60
  misc_register+0x13b/0x1c0
  sev_dev_init+0x1d4/0x290 [ccp]
  psp_dev_init+0x136/0x300 [ccp]
  sp_init+0x6f/0x80 [ccp]
  sp_pci_probe+0x2a6/0x310 [ccp]
  ? srso_alias_return_thunk+0x5/0xfbef5
  local_pci_probe+0x4b/0xb0
  work_for_cpu_fn+0x1a/0x30
  process_one_work+0x203/0x600
  worker_thread+0x19e/0x350
  ? __pfx_worker_thread+0x10/0x10
  kthread+0xeb/0x120
  ? __pfx_kthread+0x10/0x10
  ret_from_fork+0x3c/0x60
  ? __pfx_kthread+0x10/0x10
  ret_from_fork_asm+0x1a/0x30
  </TASK>
  kobject: kobject_add_internal failed for sev with -EEXIST, don't try to register things with the same name in the same directory.
  ccp 0000:22:00.1: sev initialization failed
  ccp 0000:22:00.1: psp initialization failed
  ccp 0000:a2:00.1: no command queues available
  ccp 0000:a2:00.1: psp enabled

Address this issue by unregistering the /dev/sev before clearing out
sev_data in case of PLATFORM_STATUS failure.

Fixes: 200664d5 ("crypto: ccp: Add Secure Encrypted Virtualization (SEV) command support")
Cc: stable@vger.kernel.org
Signed-off-by: Pavan Kumar Paluri <papaluri@amd.com>
Acked-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Algorithm registration is usually carried out during module init,
where as little work as possible should be carried out.  The SIMD
code violated this rule by allocating a tfm, this then triggers a
full test of the algorithm which may dead-lock in certain cases.

SIMD is only allocating the tfm to get at the alg object, which is
in fact already available as it is what we are registering.  Use
that directly and remove the crypto_alloc_tfm call.

Also remove some obsolete and unused SIMD API.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

As registration is usually carried out during module init, this
is a context where as little work as possible should be carried
out.  Testing may trigger module loads of underlying components,
which could even lead back to the module that is registering at
the moment.  This may lead to dead-locks outside of the Crypto API.

Avoid this by not waiting for the tests to complete.  They will
be scheduled but completion will be asynchronous.  Any users will
still wait for completion.
Reported-by: Russell King <linux@armlinux.org.uk>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

In order to allow testing to complete asynchronously after the
registration process, instance larvals need to complete prior
to having a test result.  Support this by redoing the lookup for
instance larvals after completion.   This should locate the pending
test larval and then repeat the wait on that (if it is still pending).

As the lookup is now repeated there is no longer any need to compute
the fulfilment status and all that code can be removed.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Use the generic crypto_authenc_extractkeys helper instead of custom
parsing code that is slightly broken.  Also fix a number of memory
leaks by moving memory allocation from setkey to init_tfm (setkey
can be called multiple times over the life of a tfm).

Finally accept all hash key lengths by running the digest over
extra-long keys.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Use the generic crypto_authenc_extractkeys helper instead of custom
parsing code that is slightly broken.  Also fix a number of memory
leaks by moving memory allocation from setkey to init_tfm (setkey
can be called multiple times over the life of a tfm).

Finally accept all hash key lengths by running the digest over
extra-long keys.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Removed CRYPTO_USED_JB and returning CRYPTO_OK instead.
Signed-off-by: Bhoomika K <bhoomikak@vayavyalabs.com>
Signed-off-by: Pavitrakumar M <pavitrakumarm@vayavyalabs.com>
Acked-by: Ruud Derwig <Ruud.Derwig@synopsys.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

This patch fixes counter width checks according to the version extension3
register. The counter widths can be 8, 16, 32 and 64 bits as per the
extension3 register.
Signed-off-by: Bhoomika K <bhoomikak@vayavyalabs.com>
Signed-off-by: Pavitrakumar M <pavitrakumarm@vayavyalabs.com>
Acked-by: Ruud Derwig <Ruud.Derwig@synopsys.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The macro `ADF_RP_INT_SRC_SEL_F_RISE_MASK` is currently set to the value
`0100b` which means "Empty Going False". This might cause an incorrect
restore of the bank state during live migration.

Fix the definition of the macro to properly represent the "Full Going
True" state which is encoded as `0011b`.

Fixes: bbfdde7d ("crypto: qat - add bank save and restore flows")
Signed-off-by: Svyatoslav Pankratov <svyatoslav.pankratov@intel.com>
Reviewed-by: Xin Zeng <xin.zeng@intel.com>
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Check for crypto_alloc_skcipher() failure.

Fixes: c8981d92 ("crypto: spacc - Add SPAcc Skcipher support")
Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The crypto_alloc_aead() function doesn't return NULL pointers, it returns
error pointers.  Fix the error checking.

Fixes: 06af76b4 ("crypto: spacc - Add SPAcc aead support")
Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Smatch complains that:

    drivers/crypto/dwc-spacc/spacc_aead.c:1031 spacc_aead_process()
    error: uninitialized symbol 'ptaadsize'.

This could happen if, for example, tctx->mode was CRYPTO_MODE_NULL and
req->cryptlen was less than icvremove.

Fixes: 06af76b4 ("crypto: spacc - Add SPAcc aead support")
Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Use the crypto_authenc_extractkeys helper rather than ad-hoc parsing.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Put the break statement should be on its own line.
Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The spacc->config.modes[] array has CRYPTO_MODE_LAST number of elements
so this > comparison should be >= to prevent an out of bounds access.

Fixes: c8981d92 ("crypto: spacc - Add SPAcc Skcipher support")
Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

This bounds checking is off by one.  The > should be >=.  The
spacc->job[] array is allocated in spacc_init() and it has
SPACC_MAX_JOBS elements.

Fixes: 8ebb14de ("crypto: spacc - Enable SPAcc AUTODETECT")
Fixes: c8981d92 ("crypto: spacc - Add SPAcc Skcipher support")
Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Print additional information, in the form of the old and new versions of
the SEV firmware, so that it can be seen what the base firmware was before
the upgrade.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Ashish Kalra <ashish.kalra@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The macros FOUR_ROUNDS_AND_SCHED and DO_4ROUNDS rely on an
unexpected/undocumented behavior of the GNU assembler, which might
change in the future
(https://sourceware.org/bugzilla/show_bug.cgi?id=32073).

    M (1) (2) // 1 arg !? Future: 2 args
    M 1 + 2   // 1 arg !? Future: 3 args

    M 1 2     // 2 args

Add parentheses around the single arguments to support future GNU
assembler and LLVM integrated assembler (when the IsOperator hack from
the following link is dropped).

Link: https://github.com/llvm/llvm-project/commit/055006475e22014b28a070db1bff41ca15f322f0Signed-off-by: Fangrui Song <maskray@google.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Add MODULE_DEVICE_TABLE(), so modules could be properly autoloaded
based on the alias from of_device_id table.
Signed-off-by: Liao Chen <liaochen4@huawei.com>
Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The user space Jitter RNG library uses the oversampling rate of 3 which
implies that each time stamp is credited with 1/3 bit of entropy. To
obtain 256 bits of entropy, 768 time stamps need to be sampled. The
increase in OSR is applied based on a report where the Jitter RNG is
used on a system exhibiting a challenging environment to collect
entropy.

This OSR default value is now applied to the Linux kernel version of
the Jitter RNG as well.

The increase in the OSR from 1 to 3 also implies that the Jitter RNG is
now slower by default.
Reported-by: Jeff Barnes <jeffbarnes@microsoft.com>
Signed-off-by: Stephan Mueller <smueller@chronox.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The custom swap functions used in octeontx2 driver do not perform any
special operations and can be replaced with the built-in swap function
of sort. This change not only reduces code size but also improves
efficiency, especially in scenarios where CONFIG_RETPOLINE is enabled,
as it makes indirect function calls more expensive.

By using the built-in swap, we avoid these costly indirect function
calls, leading to better performance.
Signed-off-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The custom swap function used in octeontx driver do not perform any
special operations and can be replaced with the built-in swap function
of sort. This change not only reduces code size but also improves
efficiency, especially in scenarios where CONFIG_RETPOLINE is enabled,
as it makes indirect function calls more expensive.

By using the built-in swap, we avoid these costly indirect function
calls, leading to better performance.
Signed-off-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Fixes: 6637e11e ("crypto: rsa - allow only odd e and restrict value in FIPS mode")
Fixes: f145d411 ("crypto: rsa - implement Chinese Remainder Theorem for faster private key operation")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Now that mpi_rshift can return errors, check them.

Fixes: 35d2bf20 ("crypto: dh - calculate Q from P for the full public key verification")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>