1. 15 Apr, 2022 6 commits
  2. 08 Apr, 2022 14 commits
    • Jakob Koschel's avatar
      crypto: cavium/nitrox - remove check of list iterator against head past the loop body · 7b2206d8
      Jakob Koschel authored
      When list_for_each_entry() completes the iteration over the whole list
      without breaking the loop, the iterator value will be a bogus pointer
      computed based on the head element.
      
      While it is safe to use the pointer to determine if it was computed
      based on the head element, either with list_entry_is_head() or
      &pos->member == head, using the iterator variable after the loop should
      be avoided.
      
      In preparation to limit the scope of a list iterator to the list
      traversal loop, use a dedicated pointer to point to the found element [1].
      
      Link: https://lore.kernel.org/all/CAHk-=wgRr_D8CB-D9Kg-c=EHreAsk5SqXPwr9Y7k9sA6cWXJ6w@mail.gmail.com/ [1]
      Signed-off-by: default avatarJakob Koschel <jakobkoschel@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      7b2206d8
    • Mario Limonciello's avatar
      crypto: ccp - When TSME and SME both detected notify user · 4e2c8794
      Mario Limonciello authored
      CC_ATTR_HOST_MEM_ENCRYPT is used to relay that memory encryption has been
      activated by the kernel.
      
      As it's technically possible to enable both SME and TSME at the same time,
      detect this scenario and notify the user that enabling TSME and SME at the
      same time is unnecessary.
      Signed-off-by: default avatarMario Limonciello <mario.limonciello@amd.com>
      Acked-by: default avatarTom Lendacky <thomas.lendacky@amd.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      4e2c8794
    • Mario Limonciello's avatar
      crypto: ccp - Allow PSP driver to load without SEV/TEE support · 84ee393b
      Mario Limonciello authored
      Previously the PSP probe routine would fail if both SEV and TEE were
      missing.  This is possibly the case for some client parts.
      
      As capabilities can now be accessed from userspace, it may still be
      useful to have the PSP driver finish loading so that those capabilities
      can be read.
      Signed-off-by: default avatarMario Limonciello <mario.limonciello@amd.com>
      Acked-by: default avatarTom Lendacky <thomas.lendacky@amd.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      84ee393b
    • Mario Limonciello's avatar
      crypto: ccp - Export PSP security bits to userspace · 50c4decc
      Mario Limonciello authored
      The PSP sets several pre-defined bits in the capabilities
      register to indicate that security attributes of the platform.
      
      Export these attributes into userspace for administrators to
      confirm platform is properly locked down.
      Acked-by: default avatarTom Lendacky <thomas.lendacky@amd.com>
      Signed-off-by: default avatarMario Limonciello <mario.limonciello@amd.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      50c4decc
    • Mario Limonciello's avatar
      crypto: ccp - cache capability into psp device · cac32cd4
      Mario Limonciello authored
      The results of the capability register will be used by future
      code at runtime rather than just initialization.
      Acked-by: default avatarTom Lendacky <thomas.lendacky@amd.com>
      Signed-off-by: default avatarMario Limonciello <mario.limonciello@amd.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      cac32cd4
    • Eric Biggers's avatar
      crypto: testmgr - test in-place en/decryption with two sglists · f17f9e90
      Eric Biggers authored
      As was established in the thread
      https://lore.kernel.org/linux-crypto/20220223080400.139367-1-gilad@benyossef.com/T/#u,
      many crypto API users doing in-place en/decryption don't use the same
      scatterlist pointers for the source and destination, but rather use
      separate scatterlists that point to the same memory.  This case isn't
      tested by the self-tests, resulting in bugs.
      
      This is the natural usage of the crypto API in some cases, so requiring
      API users to avoid this usage is not reasonable.
      
      Therefore, update the self-tests to start testing this case.
      Signed-off-by: default avatarEric Biggers <ebiggers@google.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      f17f9e90
    • Randy Dunlap's avatar
      crypto: x86 - eliminate anonymous module_init & module_exit · f16a005c
      Randy Dunlap authored
      Eliminate anonymous module_init() and module_exit(), which can lead to
      confusion or ambiguity when reading System.map, crashes/oops/bugs,
      or an initcall_debug log.
      
      Give each of these init and exit functions unique driver-specific
      names to eliminate the anonymous names.
      
      Example 1: (System.map)
       ffffffff832fc78c t init
       ffffffff832fc79e t init
       ffffffff832fc8f8 t init
      
      Example 2: (initcall_debug log)
       calling  init+0x0/0x12 @ 1
       initcall init+0x0/0x12 returned 0 after 15 usecs
       calling  init+0x0/0x60 @ 1
       initcall init+0x0/0x60 returned 0 after 2 usecs
       calling  init+0x0/0x9a @ 1
       initcall init+0x0/0x9a returned 0 after 74 usecs
      
      Fixes: 64b94cea ("crypto: blowfish - add x86_64 assembly implementation")
      Fixes: 676a3804 ("crypto: camellia-x86_64 - module init/exit functions should be static")
      Fixes: 0b95ec56 ("crypto: camellia - add assembler implementation for x86_64")
      Fixes: 56d76c96 ("crypto: serpent - add AVX2/x86_64 assembler implementation of serpent cipher")
      Fixes: b9f535ff ("[CRYPTO] twofish: i586 assembly version")
      Fixes: ff0a70fe ("crypto: twofish-x86_64-3way - module init/exit functions should be static")
      Fixes: 8280daad ("crypto: twofish - add 3-way parallel x86_64 assembler implemention")
      Signed-off-by: default avatarRandy Dunlap <rdunlap@infradead.org>
      Cc: Jussi Kivilinna <jussi.kivilinna@mbnet.fi>
      Cc: Joachim Fritschi <jfritschi@freenet.de>
      Cc: Herbert Xu <herbert@gondor.apana.org.au>
      Cc: "David S. Miller" <davem@davemloft.net>
      Cc: linux-crypto@vger.kernel.org
      Cc: x86@kernel.org
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      f16a005c
    • Hui Tang's avatar
      crypto: hisilicon/qm - optimize the barrier operation · 4cda2f4a
      Hui Tang authored
      A 'dma_wmb' barrier is enough to guarantee previous writes
      before accessing by acc device in the outer shareable domain.
      
      A 'smp_wmb' barrier is enough to guarantee previous writes
      before accessing by other cpus in the inner shareble domain.
      Signed-off-by: default avatarHui Tang <tanghui20@huawei.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      4cda2f4a
    • Yang Li's avatar
      crypto: engine - Add parameter description in crypto_transfer_request() kernel-doc comment · d5db91d2
      Yang Li authored
      Add the description of @need_pump in crypto_transfer_request() kernel-doc
      comment to remove warning found by running scripts/kernel-doc, which is
      caused by using 'make W=1'.
      
      crypto/crypto_engine.c:260: warning: Function parameter or member
      'need_pump' not described in 'crypto_transfer_request'
      Reported-by: default avatarAbaci Robot <abaci@linux.alibaba.com>
      Signed-off-by: default avatarYang Li <yang.lee@linux.alibaba.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      d5db91d2
    • Tianjia Zhang's avatar
      crypto: arm64/sm4 - add ARMv8 Crypto Extensions implementation · 5b33e0ec
      Tianjia Zhang authored
      This adds ARMv8 implementations of SM4 in ECB, CBC, CFB and CTR
      modes using Crypto Extensions, also includes key expansion operations
      because the Crypto Extensions instruction is much faster than software
      implementations.
      
      The Crypto Extensions for SM4 can only run on ARMv8 implementations
      that have support for these optional extensions.
      
      Benchmark on T-Head Yitian-710 2.75 GHz, the data comes from the 218
      mode of tcrypt. The abscissas are blocks of different lengths. The
      data is tabulated and the unit is Mb/s:
      
      sm4-generic |     16       64      128      256     1024     1420     4096
          ECB enc |  80.05    91.42    93.66    94.77    95.69    95.77    95.86
          ECB dec |  79.98    91.41    93.64    94.76    95.66    95.77    95.85
          CBC enc |  78.55    86.50    88.02    88.77    89.36    89.42    89.48
          CBC dec |  76.82    89.06    91.52    92.77    93.75    93.83    93.96
          CFB enc |  77.64    86.13    87.62    88.42    89.08    88.83    89.18
          CFB dec |  77.57    88.34    90.36    91.45    92.34    92.00    92.44
          CTR enc |  77.80    88.28    90.23    91.22    92.11    91.81    92.25
          CTR dec |  77.83    88.22    90.22    91.22    92.04    91.82    92.28
      sm4-neon
          ECB enc |  28.31   112.77   203.03   209.89   215.49   202.11   210.59
          ECB dec |  28.36   113.45   203.23   210.00   215.52   202.13   210.65
          CBC enc |  79.32    87.02    88.51    89.28    89.85    89.89    89.97
          CBC dec |  28.29   112.20   203.30   209.82   214.99   201.51   209.95
          CFB enc |  79.59    87.16    88.54    89.30    89.83    89.62    89.92
          CFB dec |  28.12   111.05   202.47   209.02   214.21   210.90   209.12
          CTR enc |  28.04   108.81   200.62   206.65   211.78   208.78   206.74
          CTR dec |  28.02   108.82   200.45   206.62   211.78   208.74   206.70
      sm4-ce-cipher
          ECB enc | 336.79   587.13   682.70   747.37   803.75   811.52   818.06
          ECB dec | 339.18   584.52   679.72   743.68   798.82   803.83   811.54
          CBC enc | 316.63   521.47   597.00   647.14   690.82   695.21   700.55
          CBC dec | 291.80   503.79   585.66   640.82   689.86   695.16   701.72
          CFB enc | 294.79   482.31   552.13   594.71   631.60   628.91   638.92
          CFB dec | 293.09   466.44   526.56   563.17   594.41   592.26   601.97
          CTR enc | 309.61   506.13   576.86   620.47   656.38   654.51   665.10
          CTR dec | 306.69   505.57   576.84   620.18   657.09   654.52   665.32
      sm4-ce
          ECB enc | 366.96  1329.81  2024.29  2755.50  3790.07  3861.91  4051.40
          ECB dec | 367.30  1323.93  2018.72  2747.43  3787.39  3862.55  4052.62
          CBC enc | 358.09   682.68   807.24   885.35   958.29   963.60   973.73
          CBC dec | 366.51  1303.63  1978.64  2667.93  3624.53  3683.41  3856.08
          CFB enc | 351.51   681.26   807.81   893.10   968.54   969.17   985.83
          CFB dec | 354.98  1266.61  1929.63  2634.81  3614.23  3611.59  3841.68
          CTR enc | 324.23  1121.25  1689.44  2256.70  2981.90  3007.79  3060.74
          CTR dec | 324.18  1120.44  1694.31  2258.32  2982.01  3010.09  3060.99
      Signed-off-by: default avatarTianjia Zhang <tianjia.zhang@linux.alibaba.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      5b33e0ec
    • Tianjia Zhang's avatar
      crypto: arm64/sm4 - add ARMv8 NEON implementation · 4f1aef9b
      Tianjia Zhang authored
      This adds ARMv8 NEON implementations of SM4 in ECB, CBC, CFB and CTR
      modes. This implementation uses the plain NEON instruction set, All
      S-BOX substitutions uses the tbl/tbx instructions of ARMv8, combined
      with the out-of-order execution in CPU, this optimization supports
      encryption of up to 8 blocks at the same time.
      
      The performance of encrypting one block is not as good as software
      implementation, so the encryption operations of CBC and CFB still
      use pure software algorithms.
      
      Benchmark on T-Head Yitian-710 2.75 GHz, the data comes from the 218
      mode of tcrypt. The abscissas are blocks of different lengths. The
      data is tabulated and the unit is Mb/s:
      
      sm4-generic |     16       64      128      256     1024     1420     4096
          ECB enc |  80.05    91.42    93.66    94.77    95.69    95.77    95.86
          ECB dec |  79.98    91.41    93.64    94.76    95.66    95.77    95.85
          CBC enc |  78.55    86.50    88.02    88.77    89.36    89.42    89.48
          CBC dec |  76.82    89.06    91.52    92.77    93.75    93.83    93.96
          CFB enc |  77.64    86.13    87.62    88.42    89.08    88.83    89.18
          CFB dec |  77.57    88.34    90.36    91.45    92.34    92.00    92.44
          CTR enc |  77.80    88.28    90.23    91.22    92.11    91.81    92.25
          CTR dec |  77.83    88.22    90.22    91.22    92.04    91.82    92.28
      sm4-neon
          ECB enc |  28.31   112.77   203.03   209.89   215.49   202.11   210.59
          ECB dec |  28.36   113.45   203.23   210.00   215.52   202.13   210.65
          CBC enc |  79.32    87.02    88.51    89.28    89.85    89.89    89.97
          CBC dec |  28.29   112.20   203.30   209.82   214.99   201.51   209.95
          CFB enc |  79.59    87.16    88.54    89.30    89.83    89.62    89.92
          CFB dec |  28.12   111.05   202.47   209.02   214.21   210.90   209.12
          CTR enc |  28.04   108.81   200.62   206.65   211.78   208.78   206.74
          CTR dec |  28.02   108.82   200.45   206.62   211.78   208.74   206.70
      Signed-off-by: default avatarTianjia Zhang <tianjia.zhang@linux.alibaba.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      4f1aef9b
    • Tianjia Zhang's avatar
      crypto: arm64/sm4-ce - rename to sm4-ce-cipher · 02436762
      Tianjia Zhang authored
      The subsequent patches of the series will have an implementation
      of SM4-ECB/CBC/CFB/CTR accelerated by the CE instruction set, which
      conflicts with the current module name. In order to keep the naming
      rules of the AES algorithm consistent, the sm4-ce algorithm is
      renamed to sm4-ce-cipher.
      
      In addition, the speed of sm4-ce-cipher is better than that of SM4
      NEON. By the way, the priority of the algorithm is adjusted to 300,
      which is also to leave room for the priority of SM4 NEON.
      Signed-off-by: default avatarTianjia Zhang <tianjia.zhang@linux.alibaba.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      02436762
    • Tianjia Zhang's avatar
      crypto: sm4 - export sm4 constant arrays · 73c919d3
      Tianjia Zhang authored
      Export the constant arrays fk, ck, sbox of the SM4 algorithm, and
      add the 'crypto_sm4_' prefix, where sbox is used in the SM4 NEON
      implementation for the tbl/tbx instruction to replace the S-BOX,
      and the fk, ck arrays are used in the SM4 CE implementation. Use
      the sm4ekey instruction to speed up key expansion operations.
      Signed-off-by: default avatarTianjia Zhang <tianjia.zhang@linux.alibaba.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      73c919d3
    • Jason A. Donenfeld's avatar
      crypto: sm3,sm4 - move into crypto directory · d2825fa9
      Jason A. Donenfeld authored
      The lib/crypto libraries live in lib because they are used by various
      drivers of the kernel. In contrast, the various helper functions in
      crypto are there because they're used exclusively by the crypto API. The
      SM3 and SM4 helper functions were erroniously moved into lib/crypto/
      instead of crypto/, even though there are no in-kernel users outside of
      the crypto API of those functions. This commit moves them into crypto/.
      
      Cc: Herbert Xu <herbert@gondor.apana.org.au>
      Cc: Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
      Cc: Eric Biggers <ebiggers@kernel.org>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      d2825fa9
  3. 03 Apr, 2022 8 commits
  4. 02 Apr, 2022 12 commits