Commit 05e1cbc4 authored by Stan Hu's avatar Stan Hu

Move Sidekiq exporter logs to log/sidekiq_exporter.log

The Sidekiq exporter logs were mixing with the normal Sidekiq logs. In order
to support structured logging in Sidekiq, we either need to split this data
out or convert the exporter to produce structured logs. Since Sidekiq job
processing is fundamentally different information from Web server traffic,
it seems cleaner to move the metrics traffic into a separate file, where they
can be parsed by a different filter if needed.

Relates to #20060
parent 6b89ab11
---
title: Move Sidekiq exporter logs to log/sidekiq_exporter.log
merge_request:
author:
type: other
...@@ -206,4 +206,12 @@ is populated whenever `gitlab-ctl reconfigure` is run manually or as part of an ...@@ -206,4 +206,12 @@ is populated whenever `gitlab-ctl reconfigure` is run manually or as part of an
Reconfigure logs files are named according to the UNIX timestamp of when the reconfigure Reconfigure logs files are named according to the UNIX timestamp of when the reconfigure
was initiated, such as `1509705644.log` was initiated, such as `1509705644.log`
## `sidekiq_exporter.log`
If Prometheus metrics and the Sidekiq Exporter are both enabled, Sidekiq will
start a Web server and listen to the defined port (default: 3807). Access logs
will be generated in `/var/log/gitlab/gitlab-rails/sidekiq_exporter.log` for
Omnibus GitLab packages or in `/home/git/gitlab/log/sidekiq_exporter.log` for
installations from source.
[repocheck]: repository_checks.md [repocheck]: repository_checks.md
...@@ -4,6 +4,8 @@ require 'prometheus/client/rack/exporter' ...@@ -4,6 +4,8 @@ require 'prometheus/client/rack/exporter'
module Gitlab module Gitlab
module Metrics module Metrics
class SidekiqMetricsExporter < Daemon class SidekiqMetricsExporter < Daemon
LOG_FILENAME = File.join(Rails.root, 'log', 'sidekiq_exporter.log')
def enabled? def enabled?
Gitlab::Metrics.metrics_folder_present? && settings.enabled Gitlab::Metrics.metrics_folder_present? && settings.enabled
end end
...@@ -17,7 +19,13 @@ module Gitlab ...@@ -17,7 +19,13 @@ module Gitlab
attr_reader :server attr_reader :server
def start_working def start_working
@server = ::WEBrick::HTTPServer.new(Port: settings.port, BindAddress: settings.address) logger = WEBrick::Log.new(LOG_FILENAME)
access_log = [
[logger, WEBrick::AccessLog::COMBINED_LOG_FORMAT]
]
@server = ::WEBrick::HTTPServer.new(Port: settings.port, BindAddress: settings.address,
Logger: logger, AccessLog: access_log)
server.mount "/", Rack::Handler::WEBrick, rack_app server.mount "/", Rack::Handler::WEBrick, rack_app
server.start server.start
end end
......
...@@ -38,7 +38,9 @@ describe Gitlab::Metrics::SidekiqMetricsExporter do ...@@ -38,7 +38,9 @@ describe Gitlab::Metrics::SidekiqMetricsExporter do
expect(::WEBrick::HTTPServer).to have_received(:new).with( expect(::WEBrick::HTTPServer).to have_received(:new).with(
Port: port, Port: port,
BindAddress: address BindAddress: address,
Logger: anything,
AccessLog: anything
) )
end end
end end
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment