memleak: expand allocator coverage (#1214)

* memleak: handle libc allocation functions other than malloc

* memleak: use tracepoints to track kernel allocations

* memleak: add combined-only mode

With large number of outstanding allocations, amount of data passed from
kernel becomes large, which slows everything down.

This patch calculates allocation statistics inside kernel, allowing user-
space part to pull combined statistics data only, thus significantly
reducing amount of passed data.

* memleak: increase hashtable capacities

There are a lot of allocations happen in kernel. Default values are not
enough to keep up.

* test: add a test for the memleak tool

man/man8/memleak.8

@@ -2,21 +2,29 @@
 .SH NAME
 memleak \- Print a summary of outstanding allocations and their call stacks to detect memory leaks. Uses Linux eBPF/bcc.
 .SH SYNOPSIS
-.B memleak [-h] [-p PID] [-t] [-a] [-o OLDER] [-c COMMAND] [-s SAMPLE_RATE]
-[-T TOP] [-z MIN_SIZE] [-Z MAX_SIZE] [-O OBJ] [INTERVAL] [COUNT]
+.B memleak [-h] [-p PID] [-t] [-a] [-o OLDER] [-c COMMAND] [--combined-only]
+[-s SAMPLE_RATE] [-T TOP] [-z MIN_SIZE] [-Z MAX_SIZE] [-O OBJ] [INTERVAL]
+[COUNT]
 .SH DESCRIPTION
 memleak traces and matches memory allocation and deallocation requests, and
 collects call stacks for each allocation. memleak can then print a summary
 of which call stacks performed allocations that weren't subsequently freed.
 
-When tracing a specific process, memleak instruments malloc and free from libc.
-When tracing all processes, memleak instruments kmalloc and kfree.
+When tracing a specific process, memleak instruments a list of allocation
+functions from libc, specifically: malloc, calloc, realloc, posix_memalign,
+valloc, memalign, pvalloc, aligned_alloc, and free.
+When tracing all processes, memleak instruments kmalloc/kfree,
+kmem_cache_alloc/kmem_cache_free, and also page allocations made by
+get_free_pages/free_pages.
 
 memleak may introduce significant overhead when tracing processes that allocate
 and free many blocks very quickly. See the OVERHEAD section below.
 
 This tool only works on Linux 4.6+. Stack traces are obtained using the new BPF_STACK_TRACE` APIs.
 For kernels older than 4.6, see the version under tools/old.
+Kernel memory allocations are intercepted through tracepoints, which are
+available on Linux 4.7+.
+
 .SH REQUIREMENTS
 CONFIG_BPF and bcc.
 .SH OPTIONS
@@ -25,7 +33,7 @@ CONFIG_BPF and bcc.
 Print usage message.
 .TP
 \-p PID
-Trace this process ID only (filtered in-kernel). This traces malloc and free from libc.
+Trace this process ID only (filtered in-kernel). This traces libc allocator.
 .TP
 \-t
 Print a trace of all allocation and free requests and results.
@@ -38,7 +46,12 @@ Print only allocations older than OLDER milliseconds. Useful to remove false pos
 The default value is 500 milliseconds.
 .TP
 \-c COMMAND
-Run the specified command and trace its allocations only. This traces malloc and free from libc.
+Run the specified command and trace its allocations only. This traces libc allocator.
+.TP
+\-\-combined-only
+Use statistics precalculated in kernel space. Amount of data to be pulled from
+kernel significantly decreases, at the cost of losing capabilities of time-based
+false positives filtering (\-o).
 .TP
 \-s SAMPLE_RATE
 Record roughly every SAMPLE_RATE-th allocation to reduce overhead.
@@ -54,7 +67,7 @@ Capture only allocations that are larger than or equal to MIN_SIZE bytes.
 Capture only allocations that are smaller than or equal to MAX_SIZE bytes.
 .TP
 \-O OBJ
-Attach to malloc and free in specified object instead of resolving libc. Ignored when kernel allocations are profiled.
+Attach to allocation functions in specified object instead of resolving libc. Ignored when kernel allocations are profiled.
 .TP
 INTERVAL
 Print a summary of oustanding allocations and their call stacks every INTERVAL seconds.
@@ -92,6 +105,10 @@ a significant slowdown. You can use the \-s switch to reduce the overhead
 further by capturing only every N-th allocation. The \-z and \-Z switches can
 also reduce overhead by capturing only allocations of specific sizes.
 
+Additionally, option \-\-combined-only saves processing time by reusing already
+calculated allocation statistics from kernel. It's faster, but lacks information
+about particular allocations.
+
 To determine the rate at which your application is calling malloc/free, or the
 rate at which your kernel is calling kmalloc/kfree, place a probe with perf and
 collect statistics. For example, to determine how many calls to __kmalloc are

tests/python/CMakeLists.txt

@@ -66,5 +66,7 @@ add_test(NAME py_test_dump_func WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
   COMMAND ${TEST_WRAPPER} py_dump_func simple ${CMAKE_CURRENT_SOURCE_DIR}/test_dump_func.py)
 add_test(NAME py_test_tools_smoke WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
   COMMAND ${TEST_WRAPPER} py_test_tools_smoke sudo ${CMAKE_CURRENT_SOURCE_DIR}/test_tools_smoke.py)
+add_test(NAME py_test_tools_memleak WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
+  COMMAND ${TEST_WRAPPER} py_test_tools_memleak sudo ${CMAKE_CURRENT_SOURCE_DIR}/test_tools_memleak.py)
 add_test(NAME py_test_usdt WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
   COMMAND ${TEST_WRAPPER} py_test_usdt sudo ${CMAKE_CURRENT_SOURCE_DIR}/test_usdt.py)

tests/python/test_tools_memleak.py

+#!/usr/bin/env python
+
+from unittest import main, skipUnless, TestCase
+import distutils.version
+import os
+import subprocess
+import sys
+import tempfile
+
+TOOLS_DIR = "../../tools/"
+
+
+class cfg:
+    cmd_format = ""
+
+    # Amount of memory to leak. Note, that test application allocates memory
+    # for its own needs in libc, so this amount should be large enough to be
+    # the biggest allocation.
+    leaking_amount = 30000
+
+
+def kernel_version_ge(major, minor):
+    # True if running kernel is >= X.Y
+    version = distutils.version.LooseVersion(os.uname()[2]).version
+    if version[0] > major:
+        return True
+    if version[0] < major:
+        return False
+    if minor and version[1] < minor:
+        return False
+    return True
+
+
+def setUpModule():
+    # Build the memory leaking application.
+    c_src = 'test_tools_memleak_leaker_app.c'
+    tmp_dir = tempfile.mkdtemp(prefix='bcc-test-memleak-')
+    c_src_full = os.path.dirname(sys.argv[0]) + os.path.sep + c_src
+    exec_dst = tmp_dir + os.path.sep + 'leaker_app'
+
+    if subprocess.call(['gcc', '-g', '-O0', '-o', exec_dst, c_src_full]) != 0:
+        print("can't compile the leaking application")
+        raise Exception
+
+    # Taking two snapshot with one second interval. Getting the largest
+    # allocation. Since attaching to a program happens with a delay, we wait
+    # for the first snapshot, then issue the command to the app. Finally,
+    # second snapshot is used to extract the information.
+    # Helper utilities "timeout" and "setbuf" are used to limit overall running
+    # time, and to disable buffering.
+    cfg.cmd_format = (
+        'stdbuf -o 0 -i 0 timeout -s KILL 10s ' + TOOLS_DIR +
+        'memleak.py -c "{} {{}} {}" -T 1 1 2'.format(exec_dst,
+                                                     cfg.leaking_amount))
+
+
+@skipUnless(kernel_version_ge(4, 6), "requires kernel >= 4.6")
+class MemleakToolTests(TestCase):
+    def run_leaker(self, leak_kind):
+        # Starting memleak.py, which in turn launches the leaking application.
+        p = subprocess.Popen(cfg.cmd_format.format(leak_kind),
+                             stdin=subprocess.PIPE, stdout=subprocess.PIPE,
+                             shell=True)
+
+        # Waiting for the first report.
+        while True:
+            p.poll()
+            if p.returncode is not None:
+                break
+            line = p.stdout.readline()
+            if "with outstanding allocations" in line:
+                break
+
+        # At this point, memleak.py have already launched application and set
+        # probes. Sending command to the leaking application to make its
+        # allocations.
+        out = p.communicate(input="\n")[0]
+
+        # If there were memory leaks, they are in the output. Filter the lines
+        # containing "byte" substring. Every interesting line is expected to
+        # start with "N bytes from"
+        x = [x for x in out.split('\n') if 'byte' in x]
+
+        self.assertTrue(len(x) >= 1,
+                        msg="At least one line should have 'byte' substring.")
+
+        # Taking last report.
+        x = x[-1].split()
+        self.assertTrue(len(x) >= 1,
+                        msg="There should be at least one word in the line.")
+
+        # First word is the leak amount in bytes.
+        return int(x[0])
+
+    def test_malloc(self):
+        self.assertEqual(cfg.leaking_amount, self.run_leaker("malloc"))
+
+    def test_calloc(self):
+        self.assertEqual(cfg.leaking_amount, self.run_leaker("calloc"))
+
+    def test_realloc(self):
+        self.assertEqual(cfg.leaking_amount, self.run_leaker("realloc"))
+
+    def test_posix_memalign(self):
+        self.assertEqual(cfg.leaking_amount, self.run_leaker("posix_memalign"))
+
+    def test_valloc(self):
+        self.assertEqual(cfg.leaking_amount, self.run_leaker("valloc"))
+
+    def test_memalign(self):
+        self.assertEqual(cfg.leaking_amount, self.run_leaker("memalign"))
+
+    def test_pvalloc(self):
+        self.assertEqual(cfg.leaking_amount, self.run_leaker("pvalloc"))
+
+    def test_aligned_alloc(self):
+        self.assertEqual(cfg.leaking_amount, self.run_leaker("aligned_alloc"))
+
+
+if __name__ == "__main__":
+    main()

tests/python/test_tools_memleak_leaker_app.c

+// This is a program that leaks memory, used for memory leak detector testing.
+
+#include <fcntl.h>
+#include <malloc.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+static void generate_leak(const char *kind, int amount) {
+  void *ptr = NULL;
+
+  if (strcmp(kind, "malloc") == 0) {
+    printf("leaking via malloc, %p\n", malloc(amount));
+    return;
+  }
+
+  if (strcmp(kind, "calloc") == 0) {
+    printf("leaking via calloc, %p\n", calloc(amount, 1));
+    return;
+  }
+
+  if (strcmp(kind, "realloc") == 0) {
+    printf("leaking via realloc, %p\n", realloc(malloc(10), amount));
+    return;
+  }
+
+  if (strcmp(kind, "posix_memalign") == 0) {
+    posix_memalign(&ptr, 512, amount);
+    printf("leaking via posix_memalign, %p\n", ptr);
+    return;
+  }
+
+  if (strcmp(kind, "valloc") == 0) {
+    printf("leaking via valloc, %p\n", valloc(amount));
+    return;
+  }
+
+  if (strcmp(kind, "memalign") == 0) {
+    printf("leaking via memalign, %p\n", memalign(512, amount));
+    return;
+  }
+
+  if (strcmp(kind, "pvalloc") == 0) {
+    printf("leaking via pvalloc, %p\n", pvalloc(amount));
+    return;
+  }
+
+  if (strcmp(kind, "aligned_alloc") == 0) {
+    printf("leaking via aligned_alloc, %p\n", aligned_alloc(512, amount));
+    return;
+  }
+
+  if (strcmp(kind, "no_leak") == 0) {
+    void *ptr = malloc(amount);
+    printf("ptr = %p\n", ptr);
+    free(ptr);
+    return;
+  }
+
+  printf("unknown leak type '%s'\n", kind);
+}
+
+int main(int argc, char *argv[]) {
+  if (argc < 2) {
+    printf("usage: leak-userspace <kind-of-leak> [amount]\n");
+    return EXIT_SUCCESS;
+  }
+
+  const char *kind = argv[1];
+
+  int amount = 30;
+  if (argc > 2) {
+    amount = atoi(argv[2]);
+    if (amount < 1)
+      amount = 1;
+  }
+
+  // Wait for something in stdin to give external detector time to attach.
+  char c;
+  read(0, &c, sizeof(c));
+
+  // Do the work.
+  generate_leak(kind, amount);
+  return EXIT_SUCCESS;
+}

tools/memleak_example.txt

@@ -7,7 +7,7 @@ of which call stacks performed allocations that weren't subsequently freed.
 For example:
 
 # ./memleak -p $(pidof allocs)
-Attaching to malloc and free in pid 5193, Ctrl+C to quit.
+Attaching to pid 5193, Ctrl+C to quit.
 [11:16:33] Top 2 stacks with outstanding allocations:
         80 bytes in 5 allocations from stack
                  main+0x6d [allocs]
@@ -33,7 +33,7 @@ stack is allocating various sizes and you want to confirm which sizes are
 prevalent. Use the -a switch:
 
 # ./memleak -p $(pidof allocs) -a
-Attaching to malloc and free in pid 5193, Ctrl+C to quit.
+Attaching to pid 5193, Ctrl+C to quit.
 [11:16:33] Top 2 stacks with outstanding allocations:
         addr = 948cd0 size = 16
         addr = 948d10 size = 16
@@ -59,12 +59,12 @@ Attaching to malloc and free in pid 5193, Ctrl+C to quit.
                  __libc_start_main+0xf0 [libc-2.21.so]
 
 
-When using the -p switch, memleak traces the allocations of a particular
-process. Without this switch, kernel allocations (kmalloc) are traced instead.
+When using the -p switch, memleak traces the libc allocations of a particular
+process. Without this switch, kernel allocations are traced instead.
 For example:
 
 # ./memleak
-Attaching to kmalloc and kfree, Ctrl+C to quit.
+Attaching to kernel allocators, Ctrl+C to quit.
 ...
         248 bytes in 4 allocations from stack
                  bpf_prog_load [kernel]
@@ -126,7 +126,7 @@ roughly 10% of the allocations and print the outstanding allocations every 5
 seconds, 3 times before quitting:
 
 # ./memleak -p $(pidof allocs) -s 10 5 3
-Attaching to malloc and free in pid 2614, Ctrl+C to quit.
+Attaching to pid 2614, Ctrl+C to quit.
 [11:16:33] Top 2 stacks with outstanding allocations:
         16 bytes in 1 allocations from stack
                  main+0x6d [allocs]
@@ -151,13 +151,14 @@ USAGE message:
 
 # ./memleak -h
 usage: memleak.py [-h] [-p PID] [-t] [-a] [-o OLDER] [-c COMMAND]
-                  [-s SAMPLE_RATE] [-T TOP] [-z MIN_SIZE] [-Z MAX_SIZE]
-                  [-O OBJ]
+                  [--combined-only] [-s SAMPLE_RATE] [-T TOP] [-z MIN_SIZE]
+                  [-Z MAX_SIZE] [-O OBJ]
                   [interval] [count]
 
 Trace outstanding memory allocations that weren't freed.
-Supports both user-mode allocations made with malloc/free and kernel-mode
-allocations made with kmalloc/kfree.
+Supports both user-mode allocations made with libc functions and kernel-mode
+allocations made with kmalloc/kmem_cache_alloc/get_free_pages and corresponding
+memory release functions.
 
 positional arguments:
   interval              interval in seconds to print outstanding allocations
@@ -175,6 +176,7 @@ optional arguments:
                         milliseconds
   -c COMMAND, --command COMMAND
                         execute and trace the specified command
+  --combined-only       show combined allocation statistics only
   -s SAMPLE_RATE, --sample-rate SAMPLE_RATE
                         sample every N-th allocation to decrease the overhead
   -T TOP, --top TOP     display only this many top allocating stacks (by size)
@@ -182,7 +184,7 @@ optional arguments:
                         capture only allocations larger than this size
   -Z MAX_SIZE, --max-size MAX_SIZE
                         capture only allocations smaller than this size
-  -O OBJ, --obj OBJ     attach to malloc & free in the specified object
+  -O OBJ, --obj OBJ     attach to allocator functions in the specified object
 
 EXAMPLES:
 
@@ -190,7 +192,7 @@ EXAMPLES:
         Trace allocations and display a summary of "leaked" (outstanding)
         allocations every 5 seconds
 ./memleak -p $(pidof allocs) -t
-        Trace allocations and display each individual call to malloc/free
+        Trace allocations and display each individual allocator function call
 ./memleak -ap $(pidof allocs) 10
         Trace allocations and display allocated addresses, sizes, and stacks
         every 10 seconds for outstanding allocations