The if pclk enable fails the refclk is not disabled.
Fix the same.
Signed-off-by: Shubhrajyoti Datta <shubhraj@xilinx.com>
Signed-off-by: Mark Brown <broonie@kernel.org>