There is no need to disable the device synchronously, so
don't do it.
Signed-off-by: Sebastian Reichel <sre@kernel.org>
Tested-by: Pavel Machek <pavel@ucw.cz>