devm_clk_get() is used so there is no reason to explicitly call
clk_put() in probe or remove functions. Also remove duplicate assign
for port->membase.

Fixes: ba44dc04 ("serial: Add Milbeaut serial control")
Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com>
Tested-by: Sugaya Taichi <sugaya.taichi@socionext.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>