This commit cleans up the background migration
we introduced in the previous milestone for orphaned deployments.

Changelog: other