If an endpoint is deleted before it's been fully added to the hardware
list, the associated qset will not be fully initialized and an oops will
occur when complete(&qset->remove_complete) is called.  This can happen
if a queued URB is cancelled.

Fix this by only removing the qset from the hardware list if the
cancelled URB had qTDs.
Signed-off-by: David Vrabel <david.vrabel@csr.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>