Merge branch 'wc-sidekiq-perf-tips' into 'master'

Docs: Add Rugged GVL issue and rbspy to Sidekiq troubleshooting docs

See merge request gitlab-org/gitlab!75269

doc/administration/troubleshooting/sidekiq.md

@@ -85,6 +85,27 @@ several `WARN` level messages. Here's an example of a single thread's backtrace:
 In some cases Sidekiq may be hung and unable to respond to the `TTIN` signal.
 Move on to other troubleshooting methods if this happens.
 
+## Ruby profiling with `rbspy`
+
+[rbspy](https://rbspy.github.io) is an easy to use and low-overhead Ruby profiler that can be used to create
+flamegraph-style diagrams of CPU usage by Ruby processes.
+
+No changes to GitLab are required to use it and it has no dependencies. To install it:
+
+1. Download the binary from the [`rbspy` releases page](https://github.com/rbspy/rbspy/releases).
+1. Make the binary executable.
+
+To profile a Sidekiq worker for one minute, run:
+
+```shell
+sudo ./rbspy record --pid <sidekiq_pid> --duration 60 --file /tmp/sidekiq_profile.svg
+```
+
+![Example rbspy flamegraph](img/sidekiq_flamegraph.png)
+
+In this example of a flamegraph generated by `rbspy`, almost all of the Sidekiq process's time is spent in `rev_parse`, a native C
+function in Rugged. In the stack, we can see `rev_parse` is being called by the `ExpirePipelineCacheWorker`.
+
 ## Process profiling with `perf`
 
 Linux has a process profiling tool called `perf` that is helpful when a certain
@@ -358,3 +379,17 @@ has number of drawbacks, as mentioned in [Why Ruby's Timeout is dangerous (and T
 > - in any of your code, regardless of whether it could have possibly raised an exception before
 >
 > Nobody writes code to defend against an exception being raised on literally any line. That's not even possible. So Thread.raise is basically like a sneak attack on your code that could result in almost anything. It would probably be okay if it were pure-functional code that did not modify any state. But this is Ruby, so that's unlikely :)
+
+## Disable Rugged
+
+Calls into Rugged, Ruby bindings for `libgit2`, [lock the Sidekiq processes's GVL](https://silverhammermba.github.io/emberb/c/#c-in-ruby-threads),
+blocking all jobs on that worker from proceeding. If Rugged calls performed by Sidekiq are slow, this can cause significant delays in
+background task processing.
+
+By default, Rugged is used when Git repository data is stored on local storage or on an NFS mount.
+[Using Rugged is recommened when using NFS](../nfs.md#improving-nfs-performance-with-gitlab), but if
+you are using local storage, disabling Rugged can improve Sidekiq performance:
+
+```shell
+sudo gitlab-rake gitlab:features:disable_rugged
+```