Check whether setting AC'97 ops succeeded and clean them
on removal so the fsl_ssi driver can be reloaded.
Signed-off-by: Maciej Szmigiero <mail@maciej.szmigiero.name>
Signed-off-by: Mark Brown <broonie@kernel.org>