This reverts commit 8dc840f8.  Christian
Pellegrin <chripell@gmail.com> reported that on some systems the patch
caused DMA to fail which is much more serious than the original skipped
audio issue.  Further investigation by Dave shows that the behaviour
depends on the clock speed of the SoC - a better fix is neeeded.
Signed-off-by: Mark Brown <broonie@opensource.wolfsonmicro.com>