That was only an extra check to tackle a few bugs around this area, now
its safe to remove it.  Replace it by an ASSERT.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>