Near the top of the function, spi_imx->controller is set to
controller (and is of course never modified again). The rest of the
function uses a mix of the two expressions.

For consistency, readability and better code generation, drop all the
spi_imx-> indirections.
Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Link: https://lore.kernel.org/r/20230425134527.483607-2-linux@rasmusvillemoes.dkSigned-off-by: Mark Brown <broonie@kernel.org>