- 20 Jan, 2023 4 commits
-
-
Koba Ko authored
The following warning appears during the CCP module re-initialization: [ 140.965403] sysfs: cannot create duplicate filename '/devices/pci0000:00/0000:00:07.1/0000:03:00.2/dma/dma0chan0' [ 140.975736] CPU: 0 PID: 388 Comm: kworker/0:2 Kdump: loaded Not tainted 6.2.0-0.rc2.18.eln124.x86_64 #1 [ 140.985185] Hardware name: HPE ProLiant DL325 Gen10/ProLiant DL325 Gen10, BIOS A41 07/17/2020 [ 140.993761] Workqueue: events work_for_cpu_fn [ 140.998151] Call Trace: [ 141.000613] <TASK> [ 141.002726] dump_stack_lvl+0x33/0x46 [ 141.006415] sysfs_warn_dup.cold+0x17/0x23 [ 141.010542] sysfs_create_dir_ns+0xba/0xd0 [ 141.014670] kobject_add_internal+0xba/0x260 [ 141.018970] kobject_add+0x81/0xb0 [ 141.022395] device_add+0xdc/0x7e0 [ 141.025822] ? complete_all+0x20/0x90 [ 141.029510] __dma_async_device_channel_register+0xc9/0x130 [ 141.035119] dma_async_device_register+0x19e/0x3b0 [ 141.039943] ccp_dmaengine_register+0x334/0x3f0 [ccp] [ 141.045042] ccp5_init+0x662/0x6a0 [ccp] [ 141.049000] ? devm_kmalloc+0x40/0xd0 [ 141.052688] ccp_dev_init+0xbb/0xf0 [ccp] [ 141.056732] ? __pci_set_master+0x56/0xd0 [ 141.060768] sp_init+0x70/0x90 [ccp] [ 141.064377] sp_pci_probe+0x186/0x1b0 [ccp] [ 141.068596] local_pci_probe+0x41/0x80 [ 141.072374] work_for_cpu_fn+0x16/0x20 [ 141.076145] process_one_work+0x1c8/0x380 [ 141.080181] worker_thread+0x1ab/0x380 [ 141.083953] ? __pfx_worker_thread+0x10/0x10 [ 141.088250] kthread+0xda/0x100 [ 141.091413] ? __pfx_kthread+0x10/0x10 [ 141.095185] ret_from_fork+0x2c/0x50 [ 141.098788] </TASK> [ 141.100996] kobject_add_internal failed for dma0chan0 with -EEXIST, don't try to register things with the same name in the same directory. [ 141.113703] ccp 0000:03:00.2: ccp initialization failed The /dma/dma0chan0 sysfs file is not removed since dma_chan object has been released in ccp_dma_release() before releasing dma device. A correct procedure would be: release dma channels first => unregister dma device => release ccp dma object. Link: https://bugzilla.kernel.org/show_bug.cgi?id=216888 Fixes: 68dbe80f ("crypto: ccp - Release dma channels before dmaengine unrgister") Tested-by:
Vladis Dronov <vdronov@redhat.com> Signed-off-by:
Koba Ko <koba.ko@canonical.com> Reviewed-by:
Tom Lendacky <thomas.lendacky@amd.com> Signed-off-by:
Herbert Xu <herbert@gondor.apana.org.au>
-
Ard Biesheuvel authored
The AEAD documentation conflates associated data and authentication tags: the former (along with the ciphertext) is authenticated by the latter. Fix the doc accordingly. Signed-off-by:
Ard Biesheuvel <ardb@kernel.org> Signed-off-by:
-
Ard Biesheuvel authored
RFC4106 wraps AES in GCM mode, and can be used with larger key sizes than 128/160 bits, just like AES itself. So add these to the tcrypt recipe so they will be benchmarked as well. Signed-off-by:
-
Ard Biesheuvel authored
Add support for RFC4106 ESP encapsulation to the accelerated GCM implementation. This results in a ~10% speedup for IPsec frames of typical size (~1420 bytes) on Cortex-A53. Signed-off-by:
-
- 18 Jan, 2023 1 commit
-
-
Herbert Xu authored
Revert the changes that added p10-aes-gcm: 0781bbd7 ("crypto: p10-aes-gcm - A perl script to process PowerPC assembler source") 41a6437a ("crypto: p10-aes-gcm - Supporting functions for ghash") 3b47ecca ("crypto: p10-aes-gcm - Supporting functions for AES") ca68a96c ("crypto: p10-aes-gcm - An accelerated AES/GCM stitched implementation") cc40379b ("crypto: p10-aes-gcm - Glue code for AES/GCM stitched implementation") 3c657e86 ("crypto: p10-aes-gcm - Update Kconfig and Makefile") These changes fail to build in many configurations and are not ready for prime time. Signed-off-by:
-
- 13 Jan, 2023 13 commits
-
-
Samuel Holland authored
At least the D1 variant requires a separate clock for the TRNG. Without this clock enabled, reading from /dev/hwrng reports: sun8i-ce 3040000.crypto: DMA timeout for TRNG (tm=96) on flow 3 Experimentation shows that the necessary clock is the SoC's internal RC oscillator. This makes sense, as noise from the oscillator can be used as a source of entropy. Signed-off-by:
Samuel Holland <samuel@sholland.org> Reviewed-by:
Jernej Skrabec <jernej.skrabec@gmail.com> Acked-by:
Corentin Labbe <clabbe.montjoie@gmail.com> Signed-off-by:
-
Samuel Holland authored
D1 has a crypto engine similar to the one in other Allwinner SoCs. Like H6, it has a separate MBUS clock gate. It also requires the internal RC oscillator to be enabled for the TRNG to return data, presumably because noise from the oscillator is used as an entropy source. This is likely the case for earlier variants as well, but it really only matters for H616 and newer SoCs, as H6 provides no way to disable the internal oscillator. Signed-off-by:
Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org> Acked-by:
-
Neal Liu authored
Add device tree binding documentation for the Aspeed Advanced High-Performance Bus (AHB) Controller. Signed-off-by:
Neal Liu <neal_liu@aspeedtech.com> Reviewed-by:
Rob Herring <robh@kernel.org> Signed-off-by:
-
Neal Liu authored
Add device tree binding documentation for the Aspeed ECDSA/RSA ACRY Engines Controller. Signed-off-by:
-
Neal Liu authored
Add acry & ahbc node to device tree for AST2600. Signed-off-by:
-
Neal Liu authored
ACRY Engine is designed to accelerate the throughput of ECDSA/RSA signature and verification. This patch aims to add ACRY RSA engine driver for hardware acceleration. Signed-off-by:
-
Ard Biesheuvel authored
The skcipher walk API implementation avoids scatterwalk_map() for mapping the source and destination buffers, and invokes kmap_atomic() directly if the buffer in question is not in low memory (which can only happen on 32-bit architectures). This avoids some overhead on 64-bit architectures, and most notably, permits the skcipher code to run with preemption enabled. Now that scatterwalk_map() has been updated to use kmap_local(), none of this is needed, so we can simply use scatterwalk_map/unmap instead. Signed-off-by:
-
Danny Tsen authored
Signed-off-by:
Danny Tsen <dtsen@linux.ibm.com> Signed-off-by:
-
Danny Tsen authored
This perl code is taken from the OpenSSL project and added gcm_init_htable function used in the p10-aes-gcm-glue.c code to initialize hash table. gcm_hash_p8 is used to hash encrypted data blocks. Signed-off-by:
-
Danny Tsen authored
This code is taken from CRYPTOGAMs[1]. The following functions are used, aes_p8_set_encrypt_key is used to generate AES round keys and aes_p8_encrypt is used to encrypt single block. Signed-off-by:
-
Danny Tsen authored
Improve overall performance of AES/GCM encrypt and decrypt operations for Power10+ CPU. Signed-off-by:
-
Danny Tsen authored
Signed-off-by:
-
Danny Tsen authored
Defined CRYPTO_P10_AES_GCM in Kconfig to support AES/GCM stitched implementation for Power10+ CPU. Added a new module driver p10-aes-gcm-crypto. Signed-off-by:
-
- 06 Jan, 2023 16 commits
-
-
Taehee Yoo authored
aria-avx512 implementation uses AVX512 and GFNI. It supports 64way parallel processing. So, byteslicing code is changed to support 64way parallel. And it exports some aria-avx2 functions such as encrypt() and decrypt(). AVX and AVX2 have 16 registers. They should use memory to store/load state because of lack of registers. But AVX512 supports 32 registers. So, it doesn't require store/load in the s-box layer. It means that it can reduce overhead of store/load in the s-box layer. Also code become much simpler. Benchmark with modprobe tcrypt mode=610 num_mb=8192, i3-12100: ARIA-AVX512(128bit and 256bit) testing speed of multibuffer ecb(aria) (ecb-aria-avx512) encryption tcrypt: 1 operation in 1504 cycles (1024 bytes) tcrypt: 1 operation in 4595 cycles (4096 bytes) tcrypt: 1 operation in 1763 cycles (1024 bytes) tcrypt: 1 operation in 5540 cycles (4096 bytes) testing speed of multibuffer ecb(aria) (ecb-aria-avx512) decryption tcrypt: 1 operation in 1502 cycles (1024 bytes) tcrypt: 1 operation in 4615 cycles (4096 bytes) tcrypt: 1 operation in 1759 cycles (1024 bytes) tcrypt: 1 operation in 5554 cycles (4096 bytes) ARIA-AVX2 with GFNI(128bit and 256bit) testing speed of multibuffer ecb(aria) (ecb-aria-avx2) encryption tcrypt: 1 operation in 2003 cycles (1024 bytes) tcrypt: 1 operation in 5867 cycles (4096 bytes) tcrypt: 1 operation in 2358 cycles (1024 bytes) tcrypt: 1 operation in 7295 cycles (4096 bytes) testing speed of multibuffer ecb(aria) (ecb-aria-avx2) decryption tcrypt: 1 operation in 2004 cycles (1024 bytes) tcrypt: 1 operation in 5956 cycles (4096 bytes) tcrypt: 1 operation in 2409 cycles (1024 bytes) tcrypt: 1 operation in 7564 cycles (4096 bytes) Signed-off-by:Taehee Yoo <ap420073@gmail.com> Signed-off-by:
-
Taehee Yoo authored
aria-avx2 implementation uses AVX2, AES-NI, and GFNI. It supports 32way parallel processing. So, byteslicing code is changed to support 32way parallel. And it exports some aria-avx functions such as encrypt() and decrypt(). There are two main logics, s-box layer and diffusion layer. These codes are the same as aria-avx implementation. But some instruction are exchanged because they don't support 256bit registers. Also, AES-NI doesn't support 256bit register. So, aesenclast and aesdeclast are used twice like below: vextracti128 $1, ymm0, xmm6; vaesenclast xmm7, xmm0, xmm0; vaesenclast xmm7, xmm6, xmm6; vinserti128 $1, xmm6, ymm0, ymm0; Benchmark with modprobe tcrypt mode=610 num_mb=8192, i3-12100: ARIA-AVX2 with GFNI(128bit and 256bit) testing speed of multibuffer ecb(aria) (ecb-aria-avx2) encryption tcrypt: 1 operation in 2003 cycles (1024 bytes) tcrypt: 1 operation in 5867 cycles (4096 bytes) tcrypt: 1 operation in 2358 cycles (1024 bytes) tcrypt: 1 operation in 7295 cycles (4096 bytes) testing speed of multibuffer ecb(aria) (ecb-aria-avx2) decryption tcrypt: 1 operation in 2004 cycles (1024 bytes) tcrypt: 1 operation in 5956 cycles (4096 bytes) tcrypt: 1 operation in 2409 cycles (1024 bytes) tcrypt: 1 operation in 7564 cycles (4096 bytes) ARIA-AVX with GFNI(128bit and 256bit) testing speed of multibuffer ecb(aria) (ecb-aria-avx) encryption tcrypt: 1 operation in 2761 cycles (1024 bytes) tcrypt: 1 operation in 9390 cycles (4096 bytes) tcrypt: 1 operation in 3401 cycles (1024 bytes) tcrypt: 1 operation in 11876 cycles (4096 bytes) testing speed of multibuffer ecb(aria) (ecb-aria-avx) decryption tcrypt: 1 operation in 2735 cycles (1024 bytes) tcrypt: 1 operation in 9424 cycles (4096 bytes) tcrypt: 1 operation in 3369 cycles (1024 bytes) tcrypt: 1 operation in 11954 cycles (4096 bytes) Signed-off-by: -
Taehee Yoo authored
aria-avx assembly code accesses members of aria_ctx with magic number offset. If the shape of struct aria_ctx is changed carelessly, aria-avx will not work. So, we need to ensure accessing members of aria_ctx with correct offset values, not with magic numbers. It adds ARIA_CTX_enc_key, ARIA_CTX_dec_key, and ARIA_CTX_rounds in the asm-offsets.c So, correct offset definitions will be generated. aria-avx assembly code can access members of aria_ctx safely with these definitions. Signed-off-by:
-
Taehee Yoo authored
avx accelerated aria module used local keystream array. But, keystream array size is too big. So, it puts the keystream array into request ctx. Signed-off-by:
-
David Rientjes authored
For SEV_GET_ID2, the user provided length does not have a specified limitation because the length of the ID may change in the future. The kernel memory allocation, however, is implicitly limited to 4MB on x86 by the page allocator, otherwise the kzalloc() will fail. When this happens, it is best not to spam the kernel log with the warning. Simply fail the allocation and return ENOMEM to the user. Fixes: d6112ea0 ("crypto: ccp - introduce SEV_GET_ID2 command") Reported-by:
Andy Nguyen <theflow@google.com> Reported-by:
Peter Gonda <pgonda@google.com> Suggested-by:
David Rientjes <rientjes@google.com> Acked-by:
-
Herbert Xu authored
GFP_DMA does not guarantee that the returned memory is aligned for DMA. It should be removed where it is superfluous. However, kmalloc may start returning DMA-unaligned memory in future so fix this by adding the alignment by hand. Signed-off-by: -
Herbert Xu authored
GFP_DMA does not guarantee that the returned memory is aligned for DMA. It should be removed where it is superfluous. However, kmalloc may start returning DMA-unaligned memory in future so fix this by adding the alignment by hand. Signed-off-by: -
Nicolai Stange authored
The kernel provides implementations of the NIST ECDSA signature verification primitives. For key sizes of 256 and 384 bits respectively they are approved and can be enabled in FIPS mode. Do so. Signed-off-by:
Nicolai Stange <nstange@suse.de> Signed-off-by:
-
Nicolai Stange authored
ghash may be used only as part of the gcm(aes) construction in FIPS mode. Since commit d6097b8d ("crypto: api - allow algs only in specific constructions in FIPS mode") there's support for using spawns which by itself are marked as non-approved from approved template instantiations. So simply mark plain ghash as non-approved in testmgr to block any attempts of direct instantiations in FIPS mode. Signed-off-by:
-
Nicolai Stange authored
cbcmac(aes) may be used only as part of the ccm(aes) construction in FIPS mode. Since commit d6097b8d ("crypto: api - allow algs only in specific constructions in FIPS mode") there's support for using spawns which by itself are marked as non-approved from approved template instantiations. So simply mark plain cbcmac(aes) as non-approved in testmgr to block any attempts of direct instantiations in FIPS mode. Signed-off-by:
-
Vladis Dronov authored
xts_fallback_setkey() in xts_aes_set_key() will now enforce key size rule in FIPS mode when setting up the fallback algorithm keys, which makes the check in xts_aes_set_key() redundant or unreachable. So just drop this check. xts_fallback_setkey() now makes a key size check in xts_verify_key(): xts_fallback_setkey() crypto_skcipher_setkey() [ skcipher_setkey_unaligned() ] cipher->setkey() { .setkey = xts_setkey } xts_setkey() xts_verify_key() Signed-off-by: -
Vladis Dronov authored
xts_check_key() is obsoleted by xts_verify_key(). Over time XTS crypto drivers adopted the newer xts_verify_key() variant, but xts_check_key() is still used by a number of drivers. Switch drivers to use the newer xts_verify_key() and make a couple of cleanups. This allows us to drop xts_check_key() completely and avoid redundancy. Signed-off-by:
Eric Biggers <ebiggers@google.com> Reviewed-by:
-
Nicolai Stange authored
According to FIPS 140-3 IG C.I., only (total) key lengths of either 256 bits or 512 bits are allowed with xts(aes). Make xts_verify_key() to reject anything else in FIPS mode. As xts(aes) is the only approved xts() template instantiation in FIPS mode, the new restriction implemented in xts_verify_key() effectively only applies to this particular construction. Signed-off-by:
-
Herbert Xu authored
GFP_DMA does not guarantee that the returned memory is aligned for DMA. In fact for sun8i-ss it is superfluous and can be removed. However, kmalloc may start returning DMA-unaligned memory in future so fix this by adding the alignment by hand. Signed-off-by:
-
Herbert Xu authored
Certain versions of gcc don't like the memcpy with a NULL dst (which only happens with a zero length). This only happens when debugging is enabled so add an if clause to work around these warnings. A similar warning used to be generated by sparse but that was fixed years ago. Link: https://lore.kernel.org/lkml/202210290446.qBayTfzl-lkp@intel.comReported-by:
kernel test robot <lkp@intel.com> Reported-by:
Kees Cook <keescook@chromium.org> Reported-by:
Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Tested-by:
-
Herbert Xu authored
The helper mpi_read_raw_from_sgl sets the number of entries in the SG list according to nbytes. However, if the last entry in the SG list contains more data than nbytes, then it may overrun the buffer because it only allocates enough memory for nbytes. Fixes: 2d4d1eea ("lib/mpi: Add mpi sgl helpers") Reported-by:
Roberto Sassu <roberto.sassu@huaweicloud.com> Signed-off-by:
-
- 30 Dec, 2022 6 commits
-
-
Herbert Xu authored
Reduce the stack usage further by splitting up the test function. Also squash blocks and unaligned_blocks into one array. Reported-by:
-
Lukas Bulwahn authored
Commit 453de3eb ("crypto: ux500/cryp - delete driver") removes the config CRYPTO_DEV_UX500_CRYP, but leaves an obsolete reference in the dependencies of config CRYPTO_DEV_UX500_DEBUG. Remove that obsolete reference, and adjust the description while at it. Fixes: 453de3eb ("crypto: ux500/cryp - delete driver") Signed-off-by:
Lukas Bulwahn <lukas.bulwahn@gmail.com> Acked-by:
Linus Walleij <linus.walleij@linaro.org> Signed-off-by:
-
Arnd Bergmann authored
The memory sanitizer causes excessive register spills in this function: crypto/wp512.c:782:13: error: stack frame size (2104) exceeds limit (2048) in 'wp512_process_buffer' [-Werror,-Wframe-larger-than] Assume that this one is safe, and mark it as needing no checks to get the stack usage back down to the normal level. Signed-off-by:
Arnd Bergmann <arnd@arndb.de> Signed-off-by:
-
Ard Biesheuvel authored
kmap_atomic() is used to create short-lived mappings of pages that may not be accessible via the kernel direct map. This is only needed on 32-bit architectures that implement CONFIG_HIGHMEM, but it can be used on 64-bit other architectures too, where the returned mapping is simply the kernel direct address of the page. However, kmap_atomic() does not support migration on CONFIG_HIGHMEM configurations, due to the use of per-CPU kmap slots, and so it disables preemption on all architectures, not just the 32-bit ones. This implies that all scatterwalk based crypto routines essentially execute with preemption disabled all the time, which is less than ideal. So let's switch scatterwalk_map/_unmap and the shash/ahash routines to kmap_local() instead, which serves a similar purpose, but without the resulting impact on preemption on architectures that have no need for CONFIG_HIGHMEM. Cc: Eric Biggers <ebiggers@kernel.org> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: "Elliott, Robert (Servers)" <elliott@hpe.com> Signed-off-by:
-
Herbert Xu authored
Instead of casting the function which upsets clang for some reason, change the assembly function siganture instead. Reported-by:
-
Sergiu Moga authored
In order for the driver to be made aware of the capabilities of the SHA and AES IP versions 0x600 , such as those present on the SAM9X60 SoC's, add a corresponding switch case to the capability method of the respective drivers. Without this, besides the capabilities not being correctly set, the self tests may hang since the driver is endlessly waiting for a completion to be set by a never occurring DMA interrupt handler. Signed-off-by:
Sergiu Moga <sergiu.moga@microchip.com> Acked-by:
Nicolas Ferre <nicolas.ferre@microchip.com> Signed-off-by:
-
