We always need the device to be runtime PM enabled to use it so just skip
the DMA initialisation not the entire prepare when polling.
Signed-off-by: Mark Brown <broonie@linaro.org>