This reverts commit eba65d17.

This broke audio on Veyron Jerry Chromebooks and I now cannot reproduce
the problem I was trying to fix even with this commit reverted, so it
seems that this was completely the wrong thing to do.
Reported-by: Enric Balletbo Serra <eballetbo@gmail.com>
Signed-off-by: John Keeping <john@metanate.com>
Signed-off-by: Mark Brown <broonie@kernel.org>