stanse found the following double lock.

In get_domain_for_dev:
  spin_lock_irqsave(&device_domain_lock, flags);
  domain_exit(domain);
    domain_remove_dev_info(domain);
      spin_lock_irqsave(&device_domain_lock, flags);
      spin_unlock_irqrestore(&device_domain_lock, flags);
  spin_unlock_irqrestore(&device_domain_lock, flags);

This happens when the domain is created by another CPU at the same time 
as this function is creating one, and the other CPU wins the race to 
attach it to the device in question, so we have to destroy our own 
newly-created one.
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>