Commit e9d4b08d authored by Stan Hu's avatar Stan Hu

Clean up stale Prometheus metrics in before_exec unicorn hook

Issuing a SIGUSR2 to a unicorn involves a dance where new workers are
spun up before the old ones are shut down. To ensure Prometheus
.db files aren't deleted after the new workers are shut down, we
need to clean these files up before the unicorn process is run.

Relates to https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/2582
parent 26c9d716
......@@ -81,6 +81,17 @@ GC.respond_to?(:copy_on_write_friendly=) and
# fast LAN.
check_client_connection false
before_exec do |server|
# The following is necessary to ensure stale Prometheus metrics don't
# accumulate over time. It needs to be done in this hook as opposed to
# inside an init script to ensure metrics files aren't deleted after new
# unicorn workers start after a SIGUSR2 is received.
if ENV['prometheus_multiproc_dir']
old_metrics = Dir[File.join(ENV['prometheus_multiproc_dir'], '*.db')]
FileUtils.rm_rf(old_metrics)
end
end
before_fork do |server, worker|
# the following is highly recommended for Rails + "preload_app true"
# as there's no need for the master process to hold a connection
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment