Commit 97ff86e0 authored by Dmitriy Zaporozhets's avatar Dmitriy Zaporozhets

Move repository when project is removed

Ths commit does next:

* When we remove project we move repository to path+deleted.git
* Then we schedule removal of path+deleted with sidekiq
* If repository move failed we abort project removal

This should help us with NFS issue when project get removed but
repository stayed. The full explanation of problem is below:

* rm -rf project.git
* rm -rf removes project.git/objects/foo
* NFS server renames foo to foo.nfsXXXX because some NFS client (think
* Unicorn) still has the file open
* rm -rf exits, but project.git/objects/foo.nfsXXX still exists
* Unicorn closes the file, the NFS client closes the file (foo), and the
* NFS server removes foo.nfsXXX
* the directory project.git/objects/ still exists => problem

So now we move repository and even if repository removal failed

Repository directory is moved so no bugs with project removed but
repository directory taken. User still able to create new project with
same name. From administrator perspective you can easily find stalled
repositories by searching `*+deleted.git`
Signed-off-by: default avatarDmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
parent d85a7437
...@@ -40,6 +40,7 @@ v 7.12.0 (unreleased) ...@@ -40,6 +40,7 @@ v 7.12.0 (unreleased)
- Add an option to automatically sign-in with an Omniauth provider - Add an option to automatically sign-in with an Omniauth provider
- Better performance for web editor (switched from satellites to rugged) - Better performance for web editor (switched from satellites to rugged)
- GitLab CI service sends .gitlab-ci.yaml in each push call - GitLab CI service sends .gitlab-ci.yaml in each push call
- When remove project - move repository and schedule it removal
v 7.11.4 v 7.11.4
- Fix missing bullets when creating lists - Fix missing bullets when creating lists
......
...@@ -97,9 +97,6 @@ class ProjectsController < ApplicationController ...@@ -97,9 +97,6 @@ class ProjectsController < ApplicationController
return access_denied! unless can?(current_user, :remove_project, @project) return access_denied! unless can?(current_user, :remove_project, @project)
::Projects::DestroyService.new(@project, current_user, {}).execute ::Projects::DestroyService.new(@project, current_user, {}).execute
respond_to do |format|
format.html do
flash[:alert] = 'Project deleted.' flash[:alert] = 'Project deleted.'
if request.referer.include?('/admin') if request.referer.include?('/admin')
...@@ -107,8 +104,8 @@ class ProjectsController < ApplicationController ...@@ -107,8 +104,8 @@ class ProjectsController < ApplicationController
else else
redirect_to dashboard_path redirect_to dashboard_path
end end
end rescue Projects::DestroyService::DestroyError => ex
end redirect_to edit_project_path(@project), alert: ex.message
end end
def autocomplete_sources def autocomplete_sources
......
module Projects module Projects
class DestroyService < BaseService class DestroyService < BaseService
include Gitlab::ShellAdapter
class DestroyError < StandardError; end
DELETED_FLAG = '+deleted'
def execute def execute
return false unless can?(current_user, :remove_project, project) return false unless can?(current_user, :remove_project, project)
project.team.truncate project.team.truncate
project.repository.expire_cache unless project.empty_repo? project.repository.expire_cache unless project.empty_repo?
if project.destroy repo_path = project.path_with_namespace
GitlabShellWorker.perform_async( wiki_path = repo_path + '.wiki'
:remove_repository,
project.path_with_namespace
)
GitlabShellWorker.perform_async( Project.transaction do
:remove_repository, project.destroy!
project.path_with_namespace + ".wiki"
)
project.satellite.destroy unless remove_repository(repo_path)
raise_error('Failed to remove project repository. Please try again or contact administrator')
end
unless remove_repository(wiki_path)
raise_error('Failed to remove wiki repository. Please try again or contact administrator')
end
end
project.satellite.destroy
log_info("Project \"#{project.name}\" was removed") log_info("Project \"#{project.name}\" was removed")
system_hook_service.execute_hooks_for(project, :destroy) system_hook_service.execute_hooks_for(project, :destroy)
true true
end end
private
def remove_repository(path)
unless gitlab_shell.exists?(path + '.git')
return true
end
new_path = removal_path(path)
if gitlab_shell.mv_repository(path, new_path)
log_info("Repository \"#{path}\" moved to \"#{new_path}\"")
GitlabShellWorker.perform_in(30.seconds, :remove_repository, new_path)
else
false
end
end
def raise_error(message)
raise DestroyError.new(message)
end
# Build a path for removing repositories
# We use `+` because its not allowed by GitLab so user can not create
# project with name cookies+119+deleted and capture someone stalled repository
#
# gitlab/cookies.git -> gitlab/cookies+119+deleted.git
#
def removal_path(path)
"#{path}+#{project.id}#{DELETED_FLAG}"
end end
end end
end end
...@@ -244,6 +244,16 @@ module Gitlab ...@@ -244,6 +244,16 @@ module Gitlab
end end
end end
# Check if such directory exists in repositories.
#
# Usage:
# exists?('gitlab')
# exists?('gitlab/cookies.git')
#
def exists?(dir_name)
File.exists?(full_path(dir_name))
end
protected protected
def gitlab_shell_path def gitlab_shell_path
...@@ -264,10 +274,6 @@ module Gitlab ...@@ -264,10 +274,6 @@ module Gitlab
File.join(repos_path, dir_name) File.join(repos_path, dir_name)
end end
def exists?(dir_name)
File.exists?(full_path(dir_name))
end
def gitlab_shell_projects_path def gitlab_shell_projects_path
File.join(gitlab_shell_path, 'bin', 'gitlab-projects') File.join(gitlab_shell_path, 'bin', 'gitlab-projects')
end end
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment