icv_ool is not used anymore, drop it.
Fixes: e345177ded ("crypto: talitos - fix AEAD processing.")
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Moves struct talitos_edesc into talitos.h so that it can be used
from any place in talitos.c
It will be required for next patch ("crypto: talitos - fix hash
on SEC1")
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Cc: stable@vger.kernel.org
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch drops the license text and replaces it
with an SPDX-License-Identifier tag.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch rewrites has_ftr_sec1() using IS_ENABLED()
instead of #ifdefs
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
There is no other file using talitos_submit in the kernel tree,
so it doesn't need to be exported nor made global.
This reverts commit 865d506155.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Fixes: 865d506155 ("crypto: talitos - export the talitos_submit function")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This driver is working well in 'simple cases', but as soon as
more exotic SG lists are provided (dst different from src,
auth part not in a single SG fragment, ...) there are
wrong results, overruns, etc ...
This patch cleans up the AEAD processing by:
- Simplifying the location of 'out of line' ICV
- Never using 'out of line' ICV on encryp
- Always using 'out of line' ICV on decrypt
- Forcing the generation of a SG table on decrypt
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Fixes: aeb4c132f3 ("crypto: talitos - Convert to new AEAD interface")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
SEC1 doesn't support S/G in descriptors so for hash operations,
the CPU has to build a buffer containing the buffered block and
the incoming data. This generates a lot of memory copies which
represents more than 50% of CPU time of a md5sum operation as
shown below with a 'perf record'.
|--86.24%-- kcapi_md_digest
| |
| |--86.18%-- _kcapi_common_vmsplice_chunk_fd
| | |
| | |--83.68%-- splice
| | | |
| | | |--83.59%-- ret_from_syscall
| | | | |
| | | | |--83.52%-- sys_splice
| | | | | |
| | | | | |--83.49%-- splice_from_pipe
| | | | | | |
| | | | | | |--83.04%-- __splice_from_pipe
| | | | | | | |
| | | | | | | |--80.67%-- pipe_to_sendpage
| | | | | | | | |
| | | | | | | | |--78.25%-- hash_sendpage
| | | | | | | | | |
| | | | | | | | | |--60.08%-- ahash_process_req
| | | | | | | | | | |
| | | | | | | | | | |--56.36%-- sg_copy_buffer
| | | | | | | | | | | |
| | | | | | | | | | | |--55.29%-- memcpy
| | | | | | | | | | | |
However, unlike SEC2+, SEC1 offers the possibility to chain
descriptors. It is therefore possible to build a first descriptor
pointing to the buffered data and a second descriptor pointing to
the incoming data, hence avoiding the memory copy to a single
buffer.
With this patch, the time necessary for a md5sum on a 90Mbytes file
is approximately 3 seconds. Without the patch it takes 6 seconds.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The number of channels is known from the beginning, no need to
test it everytime.
This patch defines two additional done functions handling only channel 0.
Then the probe registers the correct one based on the number of channels.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch zeroize the descriptor at allocation using memset().
This has two advantages:
- It reduces the number of places where data has to be set to 0
- It avoids reading memory and loading the cache with data that
will be entirely replaced.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds the following algorithms to the talitos driver:
* ecb(aes)
* ctr(aes)
* ecb(des)
* cbc(des)
* ecb(des3_ede)
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The probe error path for this driver, for all intents and purposes,
is the talitos_remove() function due to the common "goto err_out".
Without this patch applied, talitos_remove() will panic under these
two conditions:
1. If the RNG device hasn't been registered via
talitos_register_rng() prior to entry into talitos_remove(),
then the attempt to unregister the RNG "device" will cause a panic.
2. If the priv->chan array has not been allocated prior to entry
into talitos_remove(), then the per-channel FIFO cleanup will panic
because of the dereference of that NULL "array".
Both of the above scenarios occur if talitos_probe_irq() fails.
This patch resolves issue #1 by introducing a boolean to mask the
hwrng_unregister() call in talitos_unregister_rng() if RNG device
registration was unsuccessful.
It resolves issue #2 by checking that priv->chan is not NULL in the
per-channel FIFO cleanup for loop.
Signed-off-by: Aaron Sierra <asierra@xes-inc.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Compiling the talitos driver with my GCC 4.3.1 e500v2 cross-compiler
resulted in a failed build due to the anonymous union/structures
introduced in this commit:
crypto: talitos - enhanced talitos_desc struct for SEC1
The build error was:
drivers/crypto/talitos.h:56: error: unknown field 'len' specified in initializer
drivers/crypto/talitos.h:56: warning: missing braces around initializer
drivers/crypto/talitos.h:56: warning: (near initialization for 'zero_entry.<anonymous>')
drivers/crypto/talitos.h:57: error: unknown field 'j_extent' specified in initializer
drivers/crypto/talitos.h:58: error: unknown field 'eptr' specified in initializer
drivers/crypto/talitos.h:58: warning: excess elements in struct initializer
drivers/crypto/talitos.h:58: warning: (near initialization for 'zero_entry')
make[2]: *** [drivers/crypto/talitos.o] Error 1
make[1]: *** [drivers/crypto] Error 2
make: *** [drivers] Error 2
This patch eliminates the errors by relying on the C standard's
implicit assignment of zero to static variables.
Signed-off-by: Aaron Sierra <asierra@xes-inc.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
SEC1 doesn't support scatter/gather, SEC1 doesn't handle link tables.
Therefore, for SEC1 we have to do it by SW. For that, we reserve
space at the end of the extended descriptor, in lieu of the space
reserved for the link tables on SEC2, and we perform sg_copy() when
preparing the descriptors
We also adapt the max buffer size which is only 32k on SEC1 while it
is 64k on SEC2+
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adapts the interrupts handling and reset function for
SEC1. On SEC1, registers are almost similar to SEC2+, but bits
are sometimes located at different places. So we need to define
TALITOS1 and TALITOS2 versions of some fields, and manage according
to whether it is SEC1 or SEC2.
On SEC1, only one interrupt vector is dedicated to the SEC, so only
interrupt_4ch is needed.
On SEC1, interrupts are enabled by clearing related bits in IMR,
while on SEC2, interrupts are enabled by seting the bits in IMR.
SEC1 also performs parity verification in the DES Unit. We have
to disable this feature because the test vectors provided in
the kernel have parity errors.
In reset functions, only SEC2 supports continuation after error.
For SEC1, we have to reset in all cases.
For errors handling, SEC2+ names have been kept, but displayed
text have been amended to reflect exact meaning on SEC1.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
SEC 1.0, 1.2 and 2.x+ have different EU base addresses, so we need to
define pointers for each EU in the driver private data structure.
The proper address is set by the probe function depending on the
SEC type, in order to provide access to the proper address.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
SEC1 descriptor is a bit different to SEC2+ descriptor.
talitos_submit() will have to copy hdr field into hdr1 field and
send the descriptor starting at hdr1 up to next_desc.
For SEC2, it remains unchanged and next_desc is just ignored.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
We add a new feature in the features field, to mark compatible
"fsl,sec1.0"
We also define a helper function called has_ftr_sec1() to help
functions quickly determine if they are running on SEC1 or SEC2+.
When only SEC1 or SEC2 is compiled in, has_ftr_sec1() return
trivial corresponding value. If both are compiled in, feature
field is checked.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch enhances the talitos_desc struct with fields for SEC1.
SEC1 has only one header field, and has a 'next_desc' field in
addition.
This mixed descriptor will continue to fit SEC2, and for SEC1
we will recopy hdr value into hdr1 value in talitos_submit()
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
With this, now all combinations of
CBC: AES, 3DES-EDE
with
HMAC: SHA-1, SHA-224, SHA-256, SHA-384, SHA-512
are supported.
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch exports the talitos_submit function so that on
need basis same can be used by other entities.
Signed-off-by: Sandeep Malik <Sandeep.Malik@freescale.com>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch moves the talitos structure definitions from c file to its
header file so that the same can be shared on need basis.
Signed-off-by: Sandeep Malik <Sandeep.Malik@freescale.com>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Some later SEC v3.x are equipped with a second IRQ line.
By correctly assigning IRQ affinity, this feature can be
used to increase performance on dual core parts, like the
MPC8572E and P2020E.
The existence of the 2nd IRQ is determined from the device
node's interrupt property. If present, the driver remaps
two of four channels, which in turn makes those channels
trigger their interrupts on the 2nd line instead of the first.
To handle single- and dual-IRQ combinations efficiently,
talitos gets two new interrupt handlers and back-half workers.
[includes a fix to MCR_LO's address.]
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add a reg member to the channel struct and use it to
access channels.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
SEC h/w versions 2.1 and above support sha224 via explicit instruction.
Performing sha224 ahashes on earlier versions is still possible because
they support sha256 (sha224 is sha256 with different initial constants
and a different truncation length). We do this by overriding hardware
context self-initialization, and perform it manually in s/w instead.
Thanks to Lee for his fixes for correct execution on actual sec2.0 h/w.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off by: Lee Nipper <lee.nipper@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add the following alorithms to talitos:
md5,
sha1,
sha256,
sha384,
sha512.
These are all type ahash.
Signed-off-by: Lee Nipper <lee.nipper@gmail.com>
Acked-By: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Enabling extended addressing in the h/w requires we always assign the
extended address component (eptr) of the talitos h/w pointer. This is
for e500 based platforms with large memories.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Base versions handle constant folding just fine.
Signed-off-by: Harvey Harrison <harvey.harrison@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
SEC version 2.1 and above adds the capability to do the IPSec ICV
memcmp in h/w. Results of the cmp are written back in the descriptor
header, along with the done status. A new callback is added that
checks these ICCR bits instead of performing the memcmp on the core,
and is enabled by h/w capability.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
After testing on different parts, another condition was added
before using h/w auth check because different
SEC revisions require different handling.
The SEC 3.0 allows a more flexible link table where
the auth data can span separate link table entries.
The SEC 2.4/2.1 does not support this case.
So a test was added in the decrypt routine
for a fragmented case; the h/w auth check is disallowed for
revisions not having the extent in the link table;
in this case the hw auth check is done by software.
A portion of a previous change for SEC 3.0 link table handling
was removed since it became dead code with the hw auth check supported.
This seems to be the best compromise for using hw auth check
on supporting SEC revisions; it keeps the link table logic
simpler for the fragmented cases.
Signed-off-by: Lee Nipper <lee.nipper@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In talitos_interrupt, upon one done interrupt, mask further done interrupts,
and ack only any error interrupt.
In talitos_done, unmask done interrupts after completing processing.
In flush_channel, ack each done channel processed.
Keep done overflow interrupts masked because even though each pkt
is ack'ed, a few done overflows still occur.
Signed-off-by: Lee Nipper <lee.nipper@freescale.com>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds support for authenc(hmac(sha1),cbc(des3_ede))
to the talitos crypto driver for the Freescale Security Engine.
Some adjustments were made to the scatterlist to link table conversion
to make 3des work for ping -s 1439..1446.
Signed-off-by: Lee Nipper <lee.nipper@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add support for the SEC available on a wide range of PowerQUICC devices,
e.g. MPC8349E, MPC8548E.
This initial version supports authenc(hmac(sha1),cbc(aes)) for use with IPsec.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Christophe Leroy