Impact: clean up

schedule it later after disable it.
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>