From: Greg KH <greg@kroah.com>

This fixes an oops when a kobject is unregistered before it's child is.
The usb-serial devices show this bug very easily (yank out a device
while its port is opened...)

Patch was originally written by Mike Gorse <mgorse@mgorse.dhs.org>