The driver does not disable ref_clk on remove.
According to the comment, the only reason to enable the clock is to get
its rate. So, it should be safe to disable clk just after that.

By the way, clk_prepare_enable() looks to be more appropriate
than clk_enable() here.

Found by Linux Driver Verification project (linuxtesting.org).
Signed-off-by: Alexey Khoroshilov <khoroshilov@ispras.ru>
Signed-off-by: Mark Brown <broonie@kernel.org>