The Rugged/libgit2 update appears to cache the state of the
remotes. Since the remotes are deleted underneath it via Gitaly, Rugged
doesn't know to refresh the list of remotes. We now test the RPC
response and reload the Rugged repository to validate the removal.