In former commits, .private_free callback releases resources just for
data transmission. This release function can be called without the
resources are actually allocated in error paths.

This commit applies a small refactoring to clean up codes in error
paths.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Takashi Iwai <tiwai@suse.de>