1. 12 Oct, 2017 40 commits
    • LEROY Christophe's avatar
      crypto: talitos - DMA map key in setkey() · 2e13ce08
      LEROY Christophe authored
      dma_map_single() is an heavy operation which doesn't need to
      be done at each request as the key doesn't change.
      
      Instead of DMA mapping the key at every request, this patch maps it
      once in setkey()
      Signed-off-by: default avatarChristophe Leroy <christophe.leroy@c-s.fr>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      2e13ce08
    • LEROY Christophe's avatar
      crypto: talitos - simplify tests in ipsec_esp() · 9a655608
      LEROY Christophe authored
      Do (desc->hdr & DESC_HDR_TYPE_IPSEC_ESP) only once.
      Limit number of if/else paths
      Signed-off-by: default avatarChristophe Leroy <christophe.leroy@c-s.fr>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      9a655608
    • LEROY Christophe's avatar
      crypto: talitos - remove to_talitos_ptr_len() · da9de146
      LEROY Christophe authored
      to_talitos_ptr() and to_talitos_ptr_len() are always called together
      in order to fully set a ptr, so lets merge them into a single
      helper.
      Signed-off-by: default avatarChristophe Leroy <christophe.leroy@c-s.fr>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      da9de146
    • LEROY Christophe's avatar
      crypto: talitos - don't check the number of channels at each interrupt · 9c02e285
      LEROY Christophe authored
      The number of channels is known from the beginning, no need to
      test it everytime.
      This patch defines two additional done functions handling only channel 0.
      Then the probe registers the correct one based on the number of channels.
      Signed-off-by: default avatarChristophe Leroy <christophe.leroy@c-s.fr>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      9c02e285
    • LEROY Christophe's avatar
      crypto: talitos - use devm_ioremap() · fd5ea7f0
      LEROY Christophe authored
      Use devm_ioremap()
      Signed-off-by: default avatarChristophe Leroy <christophe.leroy@c-s.fr>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      fd5ea7f0
    • LEROY Christophe's avatar
      crypto: talitos - use of_property_read_u32() · fa14c6cf
      LEROY Christophe authored
      Use of_property_read_u32() to simplify DT read
      Signed-off-by: default avatarChristophe Leroy <christophe.leroy@c-s.fr>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      fa14c6cf
    • LEROY Christophe's avatar
      crypto: talitos - use devm_kmalloc() · 24b92ff2
      LEROY Christophe authored
      Replace kmalloc() by devm_kmalloc()
      Signed-off-by: default avatarChristophe Leroy <christophe.leroy@c-s.fr>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      24b92ff2
    • LEROY Christophe's avatar
      crypto: talitos - declare local functions static · 5b2cf268
      LEROY Christophe authored
      talitos_handle_buggy_hash() and talitos_sg_map() are only used
      locally, make them static
      Signed-off-by: default avatarChristophe Leroy <christophe.leroy@c-s.fr>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      5b2cf268
    • LEROY Christophe's avatar
      crypto: talitos - zeroize the descriptor with memset() · e4a647c4
      LEROY Christophe authored
      This patch zeroize the descriptor at allocation using memset().
      This has two advantages:
      - It reduces the number of places where data has to be set to 0
      - It avoids reading memory and loading the cache with data that
      will be entirely replaced.
      Signed-off-by: default avatarChristophe Leroy <christophe.leroy@c-s.fr>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      e4a647c4
    • LEROY Christophe's avatar
      crypto: talitos - fix ctr-aes-talitos · 70d355cc
      LEROY Christophe authored
      ctr-aes-talitos test fails as follows on SEC2
      
      [    0.837427] alg: skcipher: Test 1 failed (invalid result) on encryption for ctr-aes-talitos
      [    0.845763] 00000000: 16 36 d5 ee 34 f8 06 25 d7 7f 8e 56 ca 88 43 45
      [    0.852345] 00000010: f9 3f f7 17 2a b2 12 23 30 43 09 15 82 dd e1 97
      [    0.858940] 00000020: a7 f7 32 b5 eb 25 06 13 9a ec f5 29 25 f8 4d 66
      [    0.865366] 00000030: b0 03 5b 8e aa 9a 42 b6 19 33 8a e2 9d 65 96 95
      
      This patch fixes the descriptor type which is special for CTR AES
      
      Fixes: 5e75ae1b ("crypto: talitos - add new crypto modes")
      Signed-off-by: default avatarChristophe Leroy <christophe.leroy@c-s.fr>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      70d355cc
    • LEROY Christophe's avatar
      crypto: talitos - fix use of sg_link_tbl_len · fbb22137
      LEROY Christophe authored
      sg_link_tbl_len shall be used instead of cryptlen, otherwise
      SECs which perform HW CICV verification will fail.
      Signed-off-by: default avatarChristophe Leroy <christophe.leroy@c-s.fr>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      fbb22137
    • LEROY Christophe's avatar
      crypto: talitos - fix AEAD for sha224 on non sha224 capable chips · 6cda075a
      LEROY Christophe authored
      sha224 AEAD test fails with:
      
      [    2.803125] talitos ff020000.crypto: DEUISR 0x00000000_00000000
      [    2.808743] talitos ff020000.crypto: MDEUISR 0x80100000_00000000
      [    2.814678] talitos ff020000.crypto: DESCBUF 0x20731f21_00000018
      [    2.820616] talitos ff020000.crypto: DESCBUF 0x0628d64c_00000010
      [    2.826554] talitos ff020000.crypto: DESCBUF 0x0631005c_00000018
      [    2.832492] talitos ff020000.crypto: DESCBUF 0x0628d664_00000008
      [    2.838430] talitos ff020000.crypto: DESCBUF 0x061b13a0_00000080
      [    2.844369] talitos ff020000.crypto: DESCBUF 0x0631006c_00000080
      [    2.850307] talitos ff020000.crypto: DESCBUF 0x0631006c_00000018
      [    2.856245] talitos ff020000.crypto: DESCBUF 0x063100ec_00000000
      [    2.884972] talitos ff020000.crypto: failed to reset channel 0
      [    2.890503] talitos ff020000.crypto: done overflow, internal time out, or rngu error: ISR 0x20000000_00020000
      [    2.900652] alg: aead: encryption failed on test 1 for authenc-hmac-sha224-cbc-3des-talitos: ret=22
      
      This is due to SHA224 not being supported by the HW. Allthough for
      hash we are able to init the hash context by SW, it is not
      possible for AEAD. Therefore SHA224 AEAD has to be deactivated.
      Signed-off-by: default avatarChristophe Leroy <christophe.leroy@c-s.fr>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      6cda075a
    • LEROY Christophe's avatar
      crypto: talitos - fix setkey to check key weakness · f384cdc4
      LEROY Christophe authored
      Crypto manager test report the following failures:
      [    3.061081] alg: skcipher: setkey failed on test 5 for ecb-des-talitos: flags=100
      [    3.069342] alg: skcipher-ddst: setkey failed on test 5 for ecb-des-talitos: flags=100
      [    3.077754] alg: skcipher-ddst: setkey failed on test 5 for ecb-des-talitos: flags=100
      
      This is due to setkey being expected to detect weak keys.
      Signed-off-by: default avatarChristophe Leroy <christophe.leroy@c-s.fr>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      f384cdc4
    • LEROY Christophe's avatar
      crypto: talitos - fix memory corruption on SEC2 · e04a61be
      LEROY Christophe authored
      On SEC2, when using the old descriptors type (hmac snoop no afeu)
      for doing IPsec, the CICV out pointeur points out of the allocated
      memory.
      
      [    2.502554] =============================================================================
      [    2.510740] BUG dma-kmalloc-256 (Not tainted): Redzone overwritten
      [    2.516907] -----------------------------------------------------------------------------
      [    2.516907]
      [    2.526535] Disabling lock debugging due to kernel taint
      [    2.531845] INFO: 0xde858108-0xde85810b. First byte 0xf8 instead of 0xcc
      [    2.538549] INFO: Allocated in 0x806181a9 age=0 cpu=0 pid=58
      [    2.544229] 	__kmalloc+0x374/0x564
      [    2.547649] 	talitos_edesc_alloc+0x17c/0x48c
      [    2.551929] 	aead_edesc_alloc+0x80/0x154
      [    2.555863] 	aead_encrypt+0x30/0xe0
      [    2.559368] 	__test_aead+0x5a0/0x1f3c
      [    2.563042] 	test_aead+0x2c/0x110
      [    2.566371] 	alg_test_aead+0x5c/0xf4
      [    2.569958] 	alg_test+0x1dc/0x5a0
      [    2.573305] 	cryptomgr_test+0x50/0x70
      [    2.576984] 	kthread+0xd8/0x134
      [    2.580155] 	ret_from_kernel_thread+0x5c/0x64
      [    2.584534] INFO: Freed in ipsec_esp_encrypt_done+0x130/0x240 age=6 cpu=0 pid=0
      [    2.591839] 	ipsec_esp_encrypt_done+0x130/0x240
      [    2.596395] 	flush_channel+0x1dc/0x488
      [    2.600161] 	talitos2_done_4ch+0x30/0x200
      [    2.604185] 	tasklet_action+0xa0/0x13c
      [    2.607948] 	__do_softirq+0x148/0x6cc
      [    2.611623] 	irq_exit+0xc0/0x124
      [    2.614869] 	call_do_irq+0x24/0x3c
      [    2.618292] 	do_IRQ+0x78/0x108
      [    2.621369] 	ret_from_except+0x0/0x14
      [    2.625055] 	finish_task_switch+0x58/0x350
      [    2.629165] 	schedule+0x80/0x134
      [    2.632409] 	schedule_preempt_disabled+0x38/0xc8
      [    2.637042] 	cpu_startup_entry+0xe4/0x190
      [    2.641074] 	start_kernel+0x3f4/0x408
      [    2.644741] 	0x3438
      [    2.646857] INFO: Slab 0xdffbdb00 objects=9 used=1 fp=0xde8581c0 flags=0x0080
      [    2.653978] INFO: Object 0xde858008 @offset=8 fp=0xca4395df
      [    2.653978]
      [    2.661032] Redzone de858000: cc cc cc cc cc cc cc cc                          ........
      [    2.669029] Object de858008: 00 00 00 02 00 00 00 02 00 6b 6b 6b 1e 83 ea 28  .........kkk...(
      [    2.677628] Object de858018: 00 00 00 70 1e 85 80 64 ff 73 1d 21 6b 6b 6b 6b  ...p...d.s.!kkkk
      [    2.686228] Object de858028: 00 20 00 00 1e 84 17 24 00 10 00 00 1e 85 70 00  . .....$......p.
      [    2.694829] Object de858038: 00 18 00 00 1e 84 17 44 00 08 00 00 1e 83 ea 28  .......D.......(
      [    2.703430] Object de858048: 00 80 00 00 1e 84 f0 00 00 80 00 00 1e 85 70 10  ..............p.
      [    2.712030] Object de858058: 00 20 6b 00 1e 85 80 f4 6b 6b 6b 6b 00 80 02 00  . k.....kkkk....
      [    2.720629] Object de858068: 1e 84 f0 00 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b  ....kkkkkkkkkkkk
      [    2.729230] Object de858078: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b  kkkkkkkkkkkkkkkk
      [    2.737830] Object de858088: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b  kkkkkkkkkkkkkkkk
      [    2.746429] Object de858098: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b  kkkkkkkkkkkkkkkk
      [    2.755029] Object de8580a8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b  kkkkkkkkkkkkkkkk
      [    2.763628] Object de8580b8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b  kkkkkkkkkkkkkkkk
      [    2.772229] Object de8580c8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b  kkkkkkkkkkkkkkkk
      [    2.780829] Object de8580d8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b  kkkkkkkkkkkkkkkk
      [    2.789430] Object de8580e8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 73 b0 ea 9f  kkkkkkkkkkkks...
      [    2.798030] Object de8580f8: e8 18 80 d6 56 38 44 c0 db e3 4f 71 f7 ce d1 d3  ....V8D...Oq....
      [    2.806629] Redzone de858108: f8 bd 3e 4f                                      ..>O
      [    2.814279] Padding de8581b0: 5a 5a 5a 5a 5a 5a 5a 5a                          ZZZZZZZZ
      [    2.822283] CPU: 0 PID: 0 Comm: swapper Tainted: G    B           4.9.50-g995be12679 #179
      [    2.831819] Call Trace:
      [    2.834301] [dffefd20] [c01aa9a8] check_bytes_and_report+0x100/0x194 (unreliable)
      [    2.841801] [dffefd50] [c01aac3c] check_object+0x200/0x530
      [    2.847306] [dffefd80] [c01ae584] free_debug_processing+0x290/0x690
      [    2.853585] [dffefde0] [c01aec8c] __slab_free+0x308/0x628
      [    2.859000] [dffefe80] [c05057f4] ipsec_esp_encrypt_done+0x130/0x240
      [    2.865378] [dffefeb0] [c05002c4] flush_channel+0x1dc/0x488
      [    2.870968] [dffeff10] [c05007a8] talitos2_done_4ch+0x30/0x200
      [    2.876814] [dffeff30] [c002fe38] tasklet_action+0xa0/0x13c
      [    2.882399] [dffeff60] [c002f118] __do_softirq+0x148/0x6cc
      [    2.887896] [dffeffd0] [c002f954] irq_exit+0xc0/0x124
      [    2.892968] [dffefff0] [c0013adc] call_do_irq+0x24/0x3c
      [    2.898213] [c0d4be00] [c000757c] do_IRQ+0x78/0x108
      [    2.903113] [c0d4be30] [c0015c08] ret_from_except+0x0/0x14
      [    2.908634] --- interrupt: 501 at finish_task_switch+0x70/0x350
      [    2.908634]     LR = finish_task_switch+0x58/0x350
      [    2.919327] [c0d4bf20] [c085e1d4] schedule+0x80/0x134
      [    2.924398] [c0d4bf50] [c085e2c0] schedule_preempt_disabled+0x38/0xc8
      [    2.930853] [c0d4bf60] [c007f064] cpu_startup_entry+0xe4/0x190
      [    2.936707] [c0d4bfb0] [c096c434] start_kernel+0x3f4/0x408
      [    2.942198] [c0d4bff0] [00003438] 0x3438
      [    2.946137] FIX dma-kmalloc-256: Restoring 0xde858108-0xde85810b=0xcc
      [    2.946137]
      [    2.954158] FIX dma-kmalloc-256: Object at 0xde858008 not freed
      
      This patch reworks the handling of the CICV out in order
      to properly handle all cases.
      Signed-off-by: default avatarChristophe Leroy <christophe.leroy@c-s.fr>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      e04a61be
    • LEROY Christophe's avatar
      crypto: talitos - fix AEAD test failures · ec8c7d14
      LEROY Christophe authored
      AEAD tests fail when destination SG list has more than 1 element.
      
      [    2.058752] alg: aead: Test 1 failed on encryption for authenc-hmac-sha1-cbc-aes-talitos
      [    2.066965] 00000000: 53 69 6e 67 6c 65 20 62 6c 6f 63 6b 20 6d 73 67
      00000010: c0 43 ff 74 c0 43 ff e0 de 83 d1 20 de 84 8e 54
      00000020: de 83 d7 c4
      [    2.082138] alg: aead: Test 1 failed on encryption for authenc-hmac-sha1-cbc-aes-talitos
      [    2.090435] 00000000: 53 69 6e 67 6c 65 20 62 6c 6f 63 6b 20 6d 73 67
      00000010: de 84 ea 58 c0 93 1a 24 de 84 e8 59 de 84 f1 20
      00000020: 00 00 00 00
      [    2.105721] alg: aead: Test 1 failed on encryption for authenc-hmac-sha1-cbc-3des-talitos
      [    2.114259] 00000000: 6f 54 20 6f 61 4d 79 6e 53 20 63 65 65 72 73 74
      00000010: 54 20 6f 6f 4d 20 6e 61 20 79 65 53 72 63 74 65
      00000020: 20 73 6f 54 20 6f 61 4d 79 6e 53 20 63 65 65 72
      00000030: 73 74 54 20 6f 6f 4d 20 6e 61 20 79 65 53 72 63
      00000040: 74 65 20 73 6f 54 20 6f 61 4d 79 6e 53 20 63 65
      00000050: 65 72 73 74 54 20 6f 6f 4d 20 6e 61 20 79 65 53
      00000060: 72 63 74 65 20 73 6f 54 20 6f 61 4d 79 6e 53 20
      00000070: 63 65 65 72 73 74 54 20 6f 6f 4d 20 6e 61 0a 79
      00000080: c0 50 f1 ac c0 50 f3 38 c0 50 f3 94 c0 50 f5 30
      00000090: c0 99 74 3c
      [    2.166410] alg: aead: Test 1 failed on encryption for authenc-hmac-sha1-cbc-3des-talitos
      [    2.174794] 00000000: 6f 54 20 6f 61 4d 79 6e 53 20 63 65 65 72 73 74
      00000010: 54 20 6f 6f 4d 20 6e 61 20 79 65 53 72 63 74 65
      00000020: 20 73 6f 54 20 6f 61 4d 79 6e 53 20 63 65 65 72
      00000030: 73 74 54 20 6f 6f 4d 20 6e 61 20 79 65 53 72 63
      00000040: 74 65 20 73 6f 54 20 6f 61 4d 79 6e 53 20 63 65
      00000050: 65 72 73 74 54 20 6f 6f 4d 20 6e 61 20 79 65 53
      00000060: 72 63 74 65 20 73 6f 54 20 6f 61 4d 79 6e 53 20
      00000070: 63 65 65 72 73 74 54 20 6f 6f 4d 20 6e 61 0a 79
      00000080: c0 50 f1 ac c0 50 f3 38 c0 50 f3 94 c0 50 f5 30
      00000090: c0 99 74 3c
      [    2.226486] alg: No test for authenc(hmac(sha224),cbc(aes)) (authenc-hmac-sha224-cbc-aes-talitos)
      [    2.236459] alg: No test for authenc(hmac(sha224),cbc(aes)) (authenc-hmac-sha224-cbc-aes-talitos)
      [    2.247196] alg: aead: Test 1 failed on encryption for authenc-hmac-sha224-cbc-3des-talitos
      [    2.255555] 00000000: 6f 54 20 6f 61 4d 79 6e 53 20 63 65 65 72 73 74
      00000010: 54 20 6f 6f 4d 20 6e 61 20 79 65 53 72 63 74 65
      00000020: 20 73 6f 54 20 6f 61 4d 79 6e 53 20 63 65 65 72
      00000030: 73 74 54 20 6f 6f 4d 20 6e 61 20 79 65 53 72 63
      00000040: 74 65 20 73 6f 54 20 6f 61 4d 79 6e 53 20 63 65
      00000050: 65 72 73 74 54 20 6f 6f 4d 20 6e 61 20 79 65 53
      00000060: 72 63 74 65 20 73 6f 54 20 6f 61 4d 79 6e 53 20
      00000070: 63 65 65 72 73 74 54 20 6f 6f 4d 20 6e 61 0a 79
      00000080: c0 50 f1 ac c0 50 f3 38 c0 50 f3 94 c0 50 f5 30
      00000090: c0 99 74 3c c0 96 e5 b8
      [    2.309004] alg: aead: Test 1 failed on encryption for authenc-hmac-sha224-cbc-3des-talitos
      [    2.317562] 00000000: 6f 54 20 6f 61 4d 79 6e 53 20 63 65 65 72 73 74
      00000010: 54 20 6f 6f 4d 20 6e 61 20 79 65 53 72 63 74 65
      00000020: 20 73 6f 54 20 6f 61 4d 79 6e 53 20 63 65 65 72
      00000030: 73 74 54 20 6f 6f 4d 20 6e 61 20 79 65 53 72 63
      00000040: 74 65 20 73 6f 54 20 6f 61 4d 79 6e 53 20 63 65
      00000050: 65 72 73 74 54 20 6f 6f 4d 20 6e 61 20 79 65 53
      00000060: 72 63 74 65 20 73 6f 54 20 6f 61 4d 79 6e 53 20
      00000070: 63 65 65 72 73 74 54 20 6f 6f 4d 20 6e 61 0a 79
      00000080: c0 50 f1 ac c0 50 f3 38 c0 50 f3 94 c0 50 f5 30
      00000090: c0 99 74 3c c0 96 e5 b8
      [    2.370710] alg: aead: Test 1 failed on encryption for authenc-hmac-sha256-cbc-aes-talitos
      [    2.379177] 00000000: 53 69 6e 67 6c 65 20 62 6c 6f 63 6b 20 6d 73 67
      00000010: 54 20 6f 6f 4d 20 6e 61 20 79 65 53 72 63 74 65
      00000020: 20 73 6f 54 20 6f 61 4d 79 6e 53 20 63 65 65 72
      [    2.397863] alg: aead: Test 1 failed on encryption for authenc-hmac-sha256-cbc-aes-talitos
      [    2.406134] 00000000: 53 69 6e 67 6c 65 20 62 6c 6f 63 6b 20 6d 73 67
      00000010: 54 20 6f 6f 4d 20 6e 61 20 79 65 53 72 63 74 65
      00000020: 20 73 6f 54 20 6f 61 4d 79 6e 53 20 63 65 65 72
      [    2.424789] alg: aead: Test 1 failed on encryption for authenc-hmac-sha256-cbc-3des-talitos
      [    2.433491] 00000000: 6f 54 20 6f 61 4d 79 6e 53 20 63 65 65 72 73 74
      00000010: 54 20 6f 6f 4d 20 6e 61 20 79 65 53 72 63 74 65
      00000020: 20 73 6f 54 20 6f 61 4d 79 6e 53 20 63 65 65 72
      00000030: 73 74 54 20 6f 6f 4d 20 6e 61 20 79 65 53 72 63
      00000040: 74 65 20 73 6f 54 20 6f 61 4d 79 6e 53 20 63 65
      00000050: 65 72 73 74 54 20 6f 6f 4d 20 6e 61 20 79 65 53
      00000060: 72 63 74 65 20 73 6f 54 20 6f 61 4d 79 6e 53 20
      00000070: 63 65 65 72 73 74 54 20 6f 6f 4d 20 6e 61 0a 79
      00000080: c0 50 f1 ac c0 50 f3 38 c0 50 f3 94 c0 50 f5 30
      00000090: c0 99 74 3c c0 96 e5 b8 c0 96 e9 20 c0 00 3d dc
      [    2.488832] alg: aead: Test 1 failed on encryption for authenc-hmac-sha256-cbc-3des-talitos
      [    2.497387] 00000000: 6f 54 20 6f 61 4d 79 6e 53 20 63 65 65 72 73 74
      00000010: 54 20 6f 6f 4d 20 6e 61 20 79 65 53 72 63 74 65
      00000020: 20 73 6f 54 20 6f 61 4d 79 6e 53 20 63 65 65 72
      00000030: 73 74 54 20 6f 6f 4d 20 6e 61 20 79 65 53 72 63
      00000040: 74 65 20 73 6f 54 20 6f 61 4d 79 6e 53 20 63 65
      00000050: 65 72 73 74 54 20 6f 6f 4d 20 6e 61 20 79 65 53
      00000060: 72 63 74 65 20 73 6f 54 20 6f 61 4d 79 6e 53 20
      00000070: 63 65 65 72 73 74 54 20 6f 6f 4d 20 6e 61 0a 79
      00000080: c0 50 f1 ac c0 50 f3 38 c0 50 f3 94 c0 50 f5 30
      00000090: c0 99 74 3c c0 96 e5 b8 c0 96 e9 20 c0 00 3d dc
      
      This patch fixes that.
      Signed-off-by: default avatarChristophe Leroy <christophe.leroy@c-s.fr>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      ec8c7d14
    • Christian Lamparter's avatar
      crypto: crypto4xx - add aes-gcm support · 59231368
      Christian Lamparter authored
      This patch adds aes-gcm support to crypto4xx.
      Signed-off-by: default avatarChristian Lamparter <chunkeey@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      59231368
    • Christian Lamparter's avatar
      crypto: crypto4xx - add aes-ccm support · 65ea8b67
      Christian Lamparter authored
      This patch adds aes-ccm support.
      Signed-off-by: default avatarChristian Lamparter <chunkeey@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      65ea8b67
    • Christian Lamparter's avatar
      crypto: crypto4xx - prepare for AEAD support · a0aae821
      Christian Lamparter authored
      This patch enhances existing interfaces and
      functions to support AEAD ciphers in the next
      patches.
      Signed-off-by: default avatarChristian Lamparter <chunkeey@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      a0aae821
    • Christian Lamparter's avatar
      crypto: crypto4xx - simplify sa and state context acquisition · 2f77690d
      Christian Lamparter authored
      Thanks to the big overhaul of crypto4xx_build_pd(), the request-local
      sa_in, sa_out and state_record allocation can be simplified.
      
      There's no need to setup any dma coherent memory anymore and
      much of the support code can be removed.
      Signed-off-by: default avatarChristian Lamparter <chunkeey@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      2f77690d
    • Christian Lamparter's avatar
      crypto: crypto4xx - fix stalls under heavy load · 4b5b7999
      Christian Lamparter authored
      If the crypto4xx device is continuously loaded by dm-crypt
      and ipsec work, it will start to work intermittent after a
      few (between 20-30) seconds, hurting throughput and latency.
      
      This patch contains various stability improvements in order
      to fix this issue. So far, the hardware has survived more
      than a day without suffering any stalls under the continuous
      load.
      Signed-off-by: default avatarChristian Lamparter <chunkeey@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      4b5b7999
    • Christian Lamparter's avatar
      crypto: crypto4xx - fix various warnings · 64e1062b
      Christian Lamparter authored
      crypto4xx_core.c:179:6: warning: symbol 'crypto4xx_free_state_record'
      	was not declared. Should it be static?
      crypto4xx_core.c:331:5: warning: symbol 'crypto4xx_get_n_gd'
      	was not declared. Should it be static?
      crypto4xx_core.c:652:6: warning: symbol 'crypto4xx_return_pd'
      	was not declared. Should it be static?
      
      crypto4xx_return_pd() is not used by anything. Therefore it is removed.
      Signed-off-by: default avatarChristian Lamparter <chunkeey@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      64e1062b
    • Christian Lamparter's avatar
      crypto: crypto4xx - overhaul crypto4xx_build_pd() · cd4dcd6d
      Christian Lamparter authored
      This patch overhauls and fixes code related to crypto4xx_build_pd()
      
       * crypto4xx_build_pd() did not handle chained source scatterlist.
         This is fixed by replacing the buggy indexed-access of &src[idx]
         with sg_next() in the gather array setup loop.
      
       * The redundant is_hash, direction, save_iv and pd_ctl members
         in the crypto4xx_ctx struct have been removed.
          - is_hash can be derived from the crypto_async_request parameter.
          - direction is already part of the security association's
            bf.dir bitfield.
          - save_iv is unused.
          - pd_ctl always had the host_ready bit enabled anyway.
            (the hash_final case is rather pointless, since the ahash
             code has been deactivated).
      
       * make crypto4xx_build_pd()'s caller responsible for converting
         the IV to the LE32 format.
      
       * change crypto4xx_ahash_update() and crypto4xx_ahash_digest() to
         initialize a temporary destination scatterlist. This allows the
         removal of an ugly cast of req->result (which is a pointer to an
         u8-array) to a scatterlist pointer.
      
       * change crypto4xx_build_pd() return type to int. After all
         it returns -EINPROGRESS/-EBUSY.
      
       * fix crypto4xx_build_pd() thread-unsafe sa handling.
      Signed-off-by: default avatarChristian Lamparter <chunkeey@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      cd4dcd6d
    • Christian Lamparter's avatar
      crypto: crypto4xx - use the correct LE32 format for IV and key defs · 4865b122
      Christian Lamparter authored
      The hardware expects that the keys, IVs (and inner/outer hashes)
      are in the le32 format.
      
      This patch changes all hardware interface declarations to use
      the correct LE32 data format for each field.
      
      In order to pass __CHECK_ENDIAN__ checks, crypto4xx_memcpy_le
      has to be honest about the endianness of its parameters.
      The function was split and moved to the common crypto4xx_core.h
      header. This allows the compiler to generate better code if the
      sizes/len is a constant (various *_IV_LEN).
      
      Please note that the hardware isn't consistent with the endiannes
      of the save_digest field in the state record struct though.
      The hashes produced by GHASH and CBC (for CCM) will be in LE32.
      Whereas md5 and sha{1/,256,...} do not need any conversion.
      Signed-off-by: default avatarChristian Lamparter <chunkeey@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      4865b122
    • Christian Lamparter's avatar
      crypto: crypto4xx - add backlog queue support · 8ef8d195
      Christian Lamparter authored
      Previously, If the crypto4xx driver used all available
      security contexts, it would simply refuse new requests
      with -EAGAIN. CRYPTO_TFM_REQ_MAY_BACKLOG was ignored.
      
      in case of dm-crypt.c's crypt_convert() function this was
      causing the following errors to manifest, if the system was
      pushed hard enough:
      
      | EXT4-fs warning (dm-1): ext4_end_bio:314: I/O error -5 writing to ino ..
      | EXT4-fs warning (dm-1): ext4_end_bio:314: I/O error -5 writing to ino ..
      | EXT4-fs warning (dm-1): ext4_end_bio:314: I/O error -5 writing to ino ..
      | JBD2: Detected IO errors while flushing file data on dm-1-8
      | Aborting journal on device dm-1-8.
      | EXT4-fs error : ext4_journal_check_start:56: Detected aborted journal
      | EXT4-fs (dm-1): Remounting filesystem read-only
      | EXT4-fs : ext4_writepages: jbd2_start: 2048 pages, inode 498...; err -30
      
      (This did cause corruptions due to failed writes)
      
      To fix this mess, the crypto4xx driver needs to notifiy the
      user to slow down. This can be achieved by returning -EBUSY
      on requests, once the crypto hardware was falling behind.
      
      Note: -EBUSY has two different meanings. Setting the flag
      CRYPTO_TFM_REQ_MAY_BACKLOG implies that the request was
      successfully queued, by the crypto driver. To achieve this
      requirement, the implementation introduces a threshold check and
      adds logic to the completion routines in much the same way as
      AMD's Cryptographic Coprocessor (CCP) driver do.
      
      Note2: Tests showed that dm-crypt starved ipsec traffic.
      Under load, ipsec links dropped to 0 Kbits/s. This is because
      dm-crypt's callback would instantly queue the next request.
      In order to not starve ipsec, the driver reserves a small
      portion of the available crypto contexts for this purpose.
      Signed-off-by: default avatarChristian Lamparter <chunkeey@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      8ef8d195
    • Christian Lamparter's avatar
      crypto: crypto4xx - increase context and scatter ring buffer elements · 778f81d6
      Christian Lamparter authored
      If crypto4xx is used in conjunction with dm-crypt, the available
      ring buffer elements are not enough to handle the load properly.
      
      On an aes-cbc-essiv:sha256 encrypted swap partition the read
      performance is abyssal: (tested with hdparm -t)
      
      /dev/mapper/swap_crypt:
       Timing buffered disk reads:  14 MB in  3.68 seconds =   3.81 MB/sec
      
      The patch increases both PPC4XX_NUM_SD and PPC4XX_NUM_PD to 256.
      This improves the performance considerably:
      
      /dev/mapper/swap_crypt:
       Timing buffered disk reads: 104 MB in  3.03 seconds =  34.31 MB/sec
      
      Furthermore, PPC4XX_LAST_SD, PPC4XX_LAST_GD and PPC4XX_LAST_PD
      can be easily calculated from their respective PPC4XX_NUM_*
      constant.
      Signed-off-by: default avatarChristian Lamparter <chunkeey@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      778f81d6
    • Christian Lamparter's avatar
      crypto: crypto4xx - fix type mismatch compiler error · 333eb3ed
      Christian Lamparter authored
      This patch fixes a type mismatch error that I accidentally
      introduced when I moved and refactored the dynamic_contents
      helpers.
      Signed-off-by: default avatarChristian Lamparter <chunkeey@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      333eb3ed
    • Christian Lamparter's avatar
      crypto: crypto4xx - fix off-by-one AES-OFB · e9b8e4e1
      Christian Lamparter authored
      I used aes-cbc as a template for ofb. But sadly I forgot
      to update set_key method to crypto4xx_setkey_aes_ofb().
      
      this was caught by the testmgr:
      alg: skcipher: Test 1 failed (invalid result) on encr. for ofb-aes-ppc4xx
      00000000: 76 49 ab ac 81 19 b2 46 ce e9 8e 9b 12 e9 19 7d
      00000010: 50 86 cb 9b 50 72 19 ee 95 db 11 3a 91 76 78 b2
      00000020: 73 be d6 b8 e3 c1 74 3b 71 16 e6 9e 22 22 95 16
      00000030: 3f f1 ca a1 68 1f ac 09 12 0e ca 30 75 86 e1 a7
      
      With the correct set_key method, the aes-ofb cipher passes the test.
      
      name         : ofb(aes)
      driver       : ofb-aes-ppc4xx
      module       : crypto4xx
      priority     : 300
      refcnt       : 1
      selftest     : passed
      internal     : no
      type         : ablkcipher
      async        : yes
      blocksize    : 16
      min keysize  : 16
      max keysize  : 32
      ivsize       : 16
      geniv        : <default>
      Signed-off-by: default avatarChristian Lamparter <chunkeey@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      e9b8e4e1
    • Christian Lamparter's avatar
      crypto: crypto4xx - wire up hmac_mc to hmac_muting · 5a4326d3
      Christian Lamparter authored
      The hmac_mc parameter of set_dynamic_sa_command_1()
      was defined but not used. On closer inspection it
      turns out, it was never wired up.
      Signed-off-by: default avatarChristian Lamparter <chunkeey@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      5a4326d3
    • Stephan Mueller's avatar
      crypto: keywrap - simplify code · 9e49451d
      Stephan Mueller authored
      The code is simplified by using two __be64 values for the operation
      instead of using two arrays of u8. This allows to get rid of the memory
      alignment code. In addition, the crypto_xor can be replaced with a
      native XOR operation. Finally, the definition of the variables is
      re-arranged such that the data structures come before simple variables
      to potentially reduce memory space.
      Signed-off-by: default avatarStephan Mueller <smueller@chronox.de>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      9e49451d
    • Tudor-Dan Ambarus's avatar
      crypto: dh - return unsigned value for crypto_dh_key_len() · 5b3f3a8b
      Tudor-Dan Ambarus authored
      DH_KPP_SECRET_MIN_SIZE and dh_data_size() are both returning
      unsigned values.
      Signed-off-by: default avatarTudor Ambarus <tudor.ambarus@microchip.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      5b3f3a8b
    • Tudor-Dan Ambarus's avatar
      crypto: dh - return unsigned int for dh_data_size() · cb195b36
      Tudor-Dan Ambarus authored
      p->key_size, p->p_size, p->g_size are all of unsigned int type.
      Signed-off-by: default avatarTudor Ambarus <tudor.ambarus@microchip.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      cb195b36
    • Tudor-Dan Ambarus's avatar
      crypto: ecdh - return unsigned value for crypto_ecdh_key_len() · 6e97e08d
      Tudor-Dan Ambarus authored
      ECDH_KPP_SECRET_MIN_SIZE and params->key_size are both returning
      unsigned values.
      Signed-off-by: default avatarTudor Ambarus <tudor.ambarus@microchip.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      6e97e08d
    • Arvind Yadav's avatar
      crypto: bcm - pr_err() strings should end with newlines · 924c9e36
      Arvind Yadav authored
      pr_err() messages should terminated with a new-line to avoid
      other messages being concatenated onto the end.
      Signed-off-by: default avatarArvind Yadav <arvind.yadav.cs@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      924c9e36
    • Arvind Yadav's avatar
      crypto: qat - pr_err() strings should end with newlines · ed713a25
      Arvind Yadav authored
      pr_err() messages should terminated with a new-line to avoid
      other messages being concatenated onto the end.
      Signed-off-by: default avatarArvind Yadav <arvind.yadav.cs@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      ed713a25
    • Arvind Yadav's avatar
      crypto: chelsio - pr_err() strings should end with newlines · dce094ea
      Arvind Yadav authored
      pr_err() messages should terminated with a new-line to avoid
      other messages being concatenated onto the end.
      Signed-off-by: default avatarArvind Yadav <arvind.yadav.cs@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      dce094ea
    • Arvind Yadav's avatar
      crypto: virtio - pr_err() strings should end with newlines · e02b8b43
      Arvind Yadav authored
      pr_err() messages should terminated with a new-line to avoid
      other messages being concatenated onto the end.
      Signed-off-by: default avatarArvind Yadav <arvind.yadav.cs@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      e02b8b43
    • Arvind Yadav's avatar
      crypto: omap-aes - pr_err() strings should end with newlines · bfbd6254
      Arvind Yadav authored
      pr_err() messages should terminated with a new-line to avoid
      other messages being concatenated onto the end.
      Signed-off-by: default avatarArvind Yadav <arvind.yadav.cs@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      bfbd6254
    • Arvind Yadav's avatar
      hwrng: core - pr_err() strings should end with newlines · 4d0ec229
      Arvind Yadav authored
      pr_err() messages should terminated with a new-line to avoid
      other messages being concatenated onto the end.
      Signed-off-by: default avatarArvind Yadav <arvind.yadav.cs@gmail.com>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      4d0ec229
    • Christophe Jaillet's avatar
      crypto: lrw - Check for incorrect cipher name · d38efad2
      Christophe Jaillet authored
      If the cipher name does not start with 'ecb(' we should bail out, as done
      in the 'create()' function in 'crypto/xts.c'.
      
      Fixes: 700cb3f5 ("crypto: lrw - Convert to skcipher")
      Signed-off-by: default avatarChristophe JAILLET <christophe.jaillet@wanadoo.fr>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      d38efad2
    • Christophe Jaillet's avatar
      crypto: lrw - Fix an error handling path in 'create()' · 616129cc
      Christophe Jaillet authored
      All error handling paths 'goto err_drop_spawn' except this one.
      In order to avoid some resources leak, we should do it as well here.
      
      Fixes: 700cb3f5 ("crypto: lrw - Convert to skcipher")
      Signed-off-by: default avatarChristophe JAILLET <christophe.jaillet@wanadoo.fr>
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      616129cc