Remove timeout for GitalyClient remove_remote

This can be slow because it must potentially remove a lot of remote
refs (if the remote being removed has a lot of refs)

changelogs/unreleased/56015-remove-remote-timeout.yml

+---
+title: Fix removing remote mirror failure which leaves unnecessary refs behind
+merge_request: 26213
+author:
+type: fixed

lib/gitlab/gitaly_client/remote_service.rb

@@ -38,7 +38,7 @@ module Gitlab
       def remove_remote(name)
         request = Gitaly::RemoveRemoteRequest.new(repository: @gitaly_repo, name: name)
 
-        response = GitalyClient.call(@storage, :remote_service, :remove_remote, request, timeout: GitalyClient.fast_timeout)
+        response = GitalyClient.call(@storage, :remote_service, :remove_remote, request)
 
         response.result
       end