With shared reservation objects the assumption that no fence
could have been added isn't true any more.

Additional to that the BO is about to be destroyed, so removing the
fences now has no advantage whatsoever.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>