Merge branch 'mk-perfbar-memory' into 'master'

Expose memory allocations in performance bar

See merge request gitlab-org/gitlab!61332

app/assets/javascripts/performance_bar/components/performance_bar_app.vue

@@ -73,6 +73,11 @@ export default {
       header: s__('PerformanceBar|External Http calls'),
       keys: ['label', 'code', 'proxy', 'error'],
     },
+    {
+      metric: 'memory',
+      header: s__('PerformanceBar|Memory'),
+      keys: ['item_header', 'item_content'],
+    },
     {
       metric: 'total',
       header: s__('PerformanceBar|Frontend resources'),

config/initializers/peek.rb

@@ -15,9 +15,15 @@ Peek.into Peek::Views::Elasticsearch
 Peek.into Peek::Views::Rugged
 Peek.into Peek::Views::ExternalHttp
 Peek.into Peek::Views::BulletDetailed if defined?(Bullet)
+Peek.into Peek::Views::Memory
 
 Peek.into Peek::Views::Tracing if Labkit::Tracing.tracing_url_enabled?
 
+# Trigger view creation here, since views might be subscribing to Rails notifications
+# via setup_subscribers, which is called in the initializer.
+# See https://github.com/peek/peek/blob/master/lib/peek/views/view.rb
+Peek.views
+
 ActiveSupport::Notifications.subscribe('endpoint_run.grape') do |_name, _start, _finish, _id, payload|
   if request_id = payload[:env]['action_dispatch.request_id']
     Peek.adapter.save(request_id)

doc/administration/monitoring/performance/performance_bar.md

@@ -6,7 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 
 # Performance Bar **(FREE SELF)**
 
-> The **Stats** field [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/271551) in GitLab SaaS 13.9.
+> The **Stats** field [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/271551) in GitLab 13.9.
+> The **Memory** field [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330736) in GitLab 14.0.
 
 You can display the GitLab Performance Bar to see statistics for the performance
 of a page. When activated, it looks as follows:
@@ -40,9 +41,11 @@ From left to right, it displays:
     Time until something was visible to the user.
   - [**DomContentLoaded**](https://developers.google.com/web/fundamentals/performance/critical-rendering-path/measure-crp) Event.
   - **Total number of requests** the page loaded.
-- **Trace**: If Jaeger is integrated, **Trace** links to a Jaeger tracing page
+- **Memory**: the amount of memory consumed and objects allocated during the selected request.
+  Select it to display a window with more details.
+- **Trace**: if Jaeger is integrated, **Trace** links to a Jaeger tracing page
   with the current request's `correlation_id` included.
-- **+**: A link to add a request's details to the performance bar. The request
+- **+**: a link to add a request's details to the performance bar. The request
   can be added by its full URL (authenticated as the current user), or by the value of
   its `X-Request-Id` header.
 - **Download**: a link to download the raw JSON used to generate the Performance Bar reports.
@@ -52,6 +55,11 @@ From left to right, it displays:
 - **Stats** (optional): if the `GITLAB_PERFORMANCE_BAR_STATS_URL` environment variable is set,
   this URL is displayed in the bar. In GitLab 13.9 and later, used only in GitLab SaaS.
 
+NOTE:
+Not all indicators are available in all environments. For instance, the memory view
+requires to run Ruby with [specific patches](https://gitlab.com/gitlab-org/gitlab-build-images/-/blob/master/patches/ruby/2.7.2/thread-memory-allocations-2.7.patch) applied.
+When running GitLab locally using the GDK this is typically not the case and the memory view cannot be used.
+
 ## Request warnings
 
 Requests that exceed predefined limits display a warning **{warning}** icon and

lib/peek/views/memory.rb

+# frozen_string_literal: true
+
+module Peek
+  module Views
+    class Memory < View
+      MEM_TOTAL_LABEL = 'Total'
+      MEM_OBJECTS_LABEL = 'Objects allocated'
+      MEM_MALLOCS_LABEL = 'Allocator calls'
+      MEM_BYTES_LABEL = 'Large allocations'
+
+      def initialize(options = {})
+        super
+
+        @thread_memory = {}
+      end
+
+      def results
+        return thread_memory if thread_memory.empty?
+
+        {
+          calls: byte_string(thread_memory[:mem_total_bytes]),
+          summary: {
+            MEM_OBJECTS_LABEL => number_string(thread_memory[:mem_objects]),
+            MEM_MALLOCS_LABEL => number_string(thread_memory[:mem_mallocs]),
+            MEM_BYTES_LABEL => byte_string(thread_memory[:mem_bytes])
+          },
+          details: [
+            {
+              item_header: MEM_TOTAL_LABEL,
+              item_content: "Total memory use of this request. This includes both occupancy of existing heap slots " \
+                            "as well as newly allocated memory due to large objects. Not adjusted for freed memory. " \
+                            "Lower is better."
+            },
+            {
+              item_header: MEM_OBJECTS_LABEL,
+              item_content: "Total number of objects allocated by the Ruby VM during this request. " \
+                            "Not adjusted for objects that were freed again. Lower is better."
+            },
+            {
+              item_header: MEM_MALLOCS_LABEL,
+              item_content: "Total number of times Ruby had to call `malloc`, the C memory allocator. " \
+                            "This is necessary for objects that are too large to fit into a 40 Byte slot in Ruby's managed heap. " \
+                            "Lower is better."
+            },
+            {
+              item_header: MEM_BYTES_LABEL,
+              item_content: "Memory allocated for objects that did not fit into a heap slot. " \
+                            "Not adjusted for memory that was freed again. Lower is better."
+            }
+          ]
+        }
+      end
+
+      private
+
+      attr_reader :thread_memory
+
+      def setup_subscribers
+        subscribe 'process_action.action_controller' do
+          # Ensure that Peek will see memory instrumentation in `results` by triggering it when
+          # a request is done processing. Peek itself hooks into the same notification:
+          # https://github.com/peek/peek/blob/master/lib/peek/railtie.rb
+          Gitlab::InstrumentationHelper.instrument_thread_memory_allocations(thread_memory)
+        end
+      end
+
+      def byte_string(bytes)
+        ActiveSupport::NumberHelper.number_to_human_size(bytes)
+      end
+
+      def number_string(num)
+        ActiveSupport::NumberHelper.number_to_human(num, units: { thousand: 'k', million: 'M', billion: 'B' })
+      end
+    end
+  end
+end

locale/gitlab.pot

@@ -23973,6 +23973,9 @@ msgstr ""
 msgid "PerformanceBar|Gitaly calls"
 msgstr ""
 
+msgid "PerformanceBar|Memory"
+msgstr ""
+
 msgid "PerformanceBar|Redis calls"
 msgstr ""
 

spec/lib/peek/views/memory_spec.rb

+# frozen_string_literal: true
+
+require 'spec_helper'
+
+RSpec.describe Peek::Views::Memory, :request_store do
+  subject! { described_class.new }
+
+  before do
+    stub_memory_instrumentation
+  end
+
+  context 'with process_action.action_controller notification' do
+    it 'returns empty results when it has not yet fired' do
+      expect(subject.results).to eq({})
+    end
+
+    it 'returns memory instrumentation data when it has fired' do
+      publish_notification
+
+      expect(subject.results[:calls]).to eq('2 MB')
+      expect(subject.results[:details]).to all(have_key(:item_header))
+      expect(subject.results[:details]).to all(have_key(:item_content))
+      expect(subject.results[:summary]).to include('Objects allocated' => '200 k')
+      expect(subject.results[:summary]).to include('Allocator calls' => '500')
+      expect(subject.results[:summary]).to include('Large allocations' => '1 KB')
+    end
+  end
+
+  def stub_memory_instrumentation
+    start_memory = {
+      total_malloc_bytes: 1,
+      total_mallocs: 2,
+      total_allocated_objects: 3
+    }
+    allow(Gitlab::Memory::Instrumentation).to receive(:start_thread_memory_allocations).and_return(start_memory)
+    allow(Gitlab::Memory::Instrumentation).to receive(:measure_thread_memory_allocations).with(start_memory).and_return({
+      mem_total_bytes: 2_097_152,
+      mem_bytes: 1024,
+      mem_mallocs: 500,
+      mem_objects: 200_000
+    })
+    Gitlab::InstrumentationHelper.init_instrumentation_data
+  end
+
+  def publish_notification
+    headers = double
+    allow(headers).to receive(:env).and_return('action_dispatch.request_id': 'req-42')
+
+    ActiveSupport::Notifications.publish(
+      'process_action.action_controller', Time.current - 1.second, Time.current, 'id', headers: headers
+    )
+  end
+end