`/var/log/gitlab/gitlab-rails/elasticsearch.log` for Omnibus GitLab
packages or in `/home/git/gitlab/log/elasticsearch.log` for installations
from source.
This file logs information related to the Elasticsearch Integration, including
errors during indexing or searching Elasticsearch. It's stored at:
It logs information related to the Elasticsearch Integration including
errors during indexing or searching Elasticsearch.
-`/var/log/gitlab/gitlab-rails/elasticsearch.log` for Omnibus GitLab packages.
-`/home/git/gitlab/log/elasticsearch.log` for installations from source.
Each line contains a JSON line that can be ingested by services like Elasticsearch and Splunk.
Line breaks have been added to the following example line for clarity:
...
...
@@ -785,12 +788,12 @@ Line breaks have been added to the following example line for clarity:
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/17819) in GitLab 12.6.
This file lives in
`/var/log/gitlab/gitlab-rails/exceptions_json.log` for Omnibus GitLab
packages or in `/home/git/gitlab/log/exceptions_json.log` for installations
from source.
This file logs the information about exceptions being tracked by
`Gitlab::ErrorTracking`, which provides a standard and consistent way of
[processing rescued exceptions](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/development/logging.md#exception-handling). This file is stored in:
It logs the information about exceptions being tracked by `Gitlab::ErrorTracking` which provides a standard and consistent way of [processing rescued exceptions](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/development/logging.md#exception-handling).
-`/var/log/gitlab/gitlab-rails/exceptions_json.log` for Omnibus GitLab packages.
-`/home/git/gitlab/log/exceptions_json.log` for installations from source.
Each line contains a JSON line that can be ingested by Elasticsearch. For example:
...
...
@@ -820,7 +823,7 @@ Omnibus GitLab packages or in `/home/git/gitlab/log/service_measurement.log` for
installations from source.
It contains only a single structured log with measurements for each service execution.
It will contain measurements such as the number of SQL calls, `execution_time`, `gc_stats`, and `memory usage`.
It contains measurements such as the number of SQL calls, `execution_time`, `gc_stats`, and `memory usage`.
For example:
...
...
@@ -945,7 +948,7 @@ For Omnibus GitLab installations, Alertmanager logs reside in `/var/log/gitlab/a
## Crond Logs
For Omnibus GitLab installations, crond logs reside in `/var/log/gitlab/crond/`.
For Omnibus GitLab installations, `crond` logs reside in `/var/log/gitlab/crond/`.
## Grafana Logs
...
...
@@ -973,7 +976,7 @@ from a GitLab instance.
If the bug or error is readily reproducible, save the main GitLab logs
[to a file](troubleshooting/linux_cheat_sheet.md#files--dirs) while reproducing the
problem once or more times:
problem a few times:
```shell
sudo gitlab-ctl tail | tee /tmp/<case-ID-and-keywords>.log
...
...
@@ -986,7 +989,7 @@ Conclude the log gathering with <kbd>Ctrl</kbd> + <kbd>C</kbd>.
If performance degradations or cascading errors occur that can't readily be attributed to one
of the previously listed GitLab components, [GitLabSOS](https://gitlab.com/gitlab-com/support/toolbox/gitlabsos/)
can provide a perspective spanning all of Omnibus GitLab. For more details and instructions
to run it, see[the GitLabSOS documentation](https://gitlab.com/gitlab-com/support/toolbox/gitlabsos/#gitlabsos).
to run it, read[the GitLabSOS documentation](https://gitlab.com/gitlab-com/support/toolbox/gitlabsos/#gitlabsos).
NOTE: **Note:**
GitLab Support likes to use this custom-made tool.
...
...
@@ -995,5 +998,5 @@ GitLab Support likes to use this custom-made tool.
[Fast-stats](https://gitlab.com/gitlab-com/support/toolbox/fast-stats) is a tool
for creating and comparing performance statistics from GitLab logs.
For more details and instructions to run it, see
[read the documentation for fast-stats](https://gitlab.com/gitlab-com/support/toolbox/fast-stats#usage).
For more details and instructions to run it, read the
[documentation for fast-stats](https://gitlab.com/gitlab-com/support/toolbox/fast-stats#usage).