This simplifies error handling and device removal paths.
Acked-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: Dmitry Torokhov <dmitry.torokhov@gmail.com>