ADG is registering fixed rate clock for audio_clkout, but it had not
been unregister clock when removing.

Salvator-X board is using ak4613 driver now, and it supports
hw_constraints from commit 907cd880 ("ASoC: ak4613: add
hw_constraint rule for Sampling Rate").
And this calculation is using input clk. This ak4613 input clock is
ADG clkout on Salvator-X.

Because ADG had not been unregister clkout when unbinding, it receives
fixed rate clk register error when re-binding.
Thus, ak4613 can't get correct input clock, and hw_constraints will be
failed after re-binding.
This means Salvator-X board can't use sound after unbind/bind.

This patch solves this issue.
Signed-off-by: Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: Mark Brown <broonie@kernel.org>