The config.speed_hz is just setted at the two lines ahead of it,
obviously we don't need to copy the code to set the config.speed_hz
again. Remove it for a cleanup.

Cc: Mark Brown <broonie@opensource.wolfsonmicro.com>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Hui Wang <jason77.wang@gmail.com>
Signed-off-by: Mark Brown <broonie@opensource.wolfsonmicro.com>