execsnoop

README.md

@@ -71,6 +71,7 @@ Tools:
 - tools/[biosnoop](tools/biosnoop.py): Trace block device I/O with PID and latency. [Examples](tools/biosnoop_example.txt).
 - tools/[bitesize](tools/bitesize.py): Show per process I/O size histogram. [Examples](tools/bitesize_example.txt).
 - tools/[cachestat](tools/cachestat.py): Trace page cache hit/miss ratio. [Examples](tools/cachestat_example.txt).
+- tools/[execsnoop](tools/execsnoop.py): Trace new processes via exec() syscalls.. [Examples](tools/execsnoop_example.txt).
 - tools/[fsslower](tools/fsslower.py): Trace slow file system synchronous reads and writes. [Examples](tools/fsslower_example.txt).
 - tools/[funccount](tools/funccount.py): Count kernel function calls. [Examples](tools/funccount_example.txt).
 - tools/[funclatency](tools/funclatency.py): Time kernel functions and show their latency distribution. [Examples](tools/funclatency_example.txt).

man/man8/execsnoop.8

+.TH execsnoop 8  "2016-02-07" "USER COMMANDS"
+.SH NAME
+execsnoop \- Trace new processes via exec() syscalls. Uses Linux eBPF/bcc.
+.SH SYNOPSIS
+.B execsnoop [\-h] [\-t] [\-X] [\-n NAME]
+.SH DESCRIPTION
+execsnoop traces new processes, showing the filename executed, argument
+list, and return value (0 for success).
+
+It works by traces the execve() system call (commonly used exec() variant).
+This catches new processes that follow the fork->exec sequence, as well as
+processes that re-exec() themselves. Some applications fork() but do not
+exec(), eg, for worker processes, which won't be included in the execsnoop
+output.
+
+This works by tracing the kernel sys_execve() function using dynamic tracing,
+and will need updating to match any changes to this function.
+
+Since this uses BPF, only the root user can use this tool.
+.SH REQUIREMENTS
+CONFIG_BPF and bcc.
+.SH OPTIONS
+.TP
+\-h
+Print usage message.
+.TP
+\-t
+Include a timestamp column.
+.TP
+\-X
+Exclude failed exec()s
+.TP
+\-n NAME
+Only print command lines matching this name (regex), matched anywhere
+.SH EXAMPLES
+.TP
+Trace all exec() syscalls:
+#
+.B execsnoop
+.TP
+Trace all exec() syscalls, and include timestamps:
+#
+.B execsnoop \-t
+.TP
+Only trace successful exec()s:
+#
+.B execsnoop \-X
+.TP
+Only trace exec()s where the filename or arguments contain "mount":
+#
+.B opensnoop \-n mount
+.SH FIELDS
+.TP
+TIME(s)
+Time of exec() return, in seconds.
+.TP
+PCOMM
+Parent process/command name.
+.TP
+PID
+Process ID
+.TP
+RET
+Return value of exec(). 0 == successs.
+.TP
+ARGS
+Filename for the exec(), followed be up to 19 arguments. An ellipsis "..." is
+shown if the argument list is known to be truncated.
+.SH OVERHEAD
+This traces the kernel execve function and prints output for each event. As the
+rate of this is generally expected to be low (< 1000/s), the overhead is also
+expected to be negligible. If you have an application that is calling a high
+rate of exec()s, then test and understand overhead before use.
+.SH SOURCE
+This is from bcc.
+.IP
+https://github.com/iovisor/bcc
+.PP
+Also look in the bcc distribution for a companion _examples.txt file containing
+example usage, output, and commentary for this tool.
+.SH OS
+Linux
+.SH STABILITY
+Unstable - in development.
+.SH AUTHOR
+Brendan Gregg
+.SH SEE ALSO
+opensnoop(1)

tools/execsnoop.py

+#!/usr/bin/python
+# @lint-avoid-python-3-compatibility-imports
+#
+# execsnoop Trace new processes via exec() syscalls.
+#           For Linux, uses BCC, eBPF. Embedded C.
+#
+# USAGE: execsnoop [-h] [-t] [-X] [-n NAME]
+#
+# This currently will print up to a maximum of 19 arguments, plus the process
+# name, so 20 fields in total (MAXARG).
+#
+# This won't catch all new processes: an application may fork() but not exec().
+#
+# Copyright 2016 Netflix, Inc.
+# Licensed under the Apache License, Version 2.0 (the "License")
+#
+# 07-Feb-2016   Brendan Gregg   Created this.
+
+from __future__ import print_function
+from bcc import BPF
+import argparse
+import re
+
+# arguments
+examples = """examples:
+    ./execsnoop           # trace all exec() syscalls
+    ./execsnoop -X        # only show successful exec()s
+    ./execsnoop -t        # include timestamps
+    ./execsnoop -n main   # only print command lines containing "main"
+"""
+parser = argparse.ArgumentParser(
+    description="Trace exec() syscalls",
+    formatter_class=argparse.RawDescriptionHelpFormatter,
+    epilog=examples)
+parser.add_argument("-t", "--timestamp", action="store_true",
+    help="include timestamp on output")
+parser.add_argument("-X", "--excludefails", action="store_true",
+    help="exclude failed exec()s")
+parser.add_argument("-n", "--name",
+    help="only print commands matching this name (regex), any arg")
+args = parser.parse_args()
+
+# define BPF program
+bpf_text = """
+#include <uapi/linux/ptrace.h>
+#include <linux/sched.h>
+#include <linux/fs.h>
+
+#define MAXARG   20
+#define ARGSIZE  64
+
+static int print_arg(void *ptr) {
+    // Fetch an argument, and print using bpf_trace_printk(). This is a work
+    // around until we have a binary trace interface for passing event data to
+    // bcc. Since exec()s should be low frequency, the additional overhead in
+    // this case should not be a problem.
+    const char *argp = NULL;
+    char buf[ARGSIZE] = {};
+
+    bpf_probe_read(&argp, sizeof(argp), ptr);
+    if (argp == NULL) return 0;
+
+    bpf_probe_read(&buf, sizeof(buf), (void *)(argp));
+    bpf_trace_printk("ARG %s\\n", buf);
+
+    return 1;
+}
+
+int kprobe__sys_execve(struct pt_regs *ctx, struct filename *filename,
+    const char __user *const __user *__argv,
+    const char __user *const __user *__envp)
+{
+    char fname[ARGSIZE] = {};
+    bpf_probe_read(&fname, sizeof(fname), (void *)(filename));
+    bpf_trace_printk("ARG %s\\n", fname);
+
+    int i = 1;  // skip first arg, as we printed fname
+
+    // unrolled loop to walk argv[] (MAXARG)
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++;
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++;
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++;
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++;
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++;
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++;
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++;
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++;
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++;
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++; // X
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++;
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++;
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++;
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++;
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++;
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++;
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++;
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++;
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++;
+    if (print_arg((void *)&__argv[i]) == 0) goto out; i++; // XX
+    bpf_trace_printk("ARG ...\\n");    // truncated
+
+out:
+    return 0;
+}
+
+int kretprobe__sys_execve(struct pt_regs *ctx)
+{
+    bpf_trace_printk("RET %d\\n", ctx->ax);
+    return 0;
+}
+"""
+
+# initialize BPF
+b = BPF(text=bpf_text)
+
+# header
+if args.timestamp:
+    print("%-8s" % ("TIME(s)"), end="")
+print("%-16s %-6s %3s %s" % ("PCOMM", "PID", "RET", "ARGS"))
+
+start_ts = 0
+cmd = {}
+pcomm = {}
+
+# format output
+while 1:
+    (task, pid, cpu, flags, ts, msg) = b.trace_fields()
+    (type, arg) = msg.split(" ", 1)
+
+    if start_ts == 0:
+        start_ts = ts
+
+    if type == "RET":
+        skip = 0
+        if args.name:
+            if not re.search(args.name, cmd[pid]):
+                skip = 1
+        if args.excludefails and int(arg) < 0:
+            skip = 1
+        if skip:
+            del cmd[pid]
+            del pcomm[pid]
+            continue
+
+        # output
+        if args.timestamp:
+            print("%-8.3f" % (ts - start_ts), end="")
+        print("%-16s %-6s %3s %s" % (pcomm[pid], pid, arg, cmd[pid]))
+        del cmd[pid]
+        del pcomm[pid]
+    else:
+        # build command line string
+        if pid in cmd:
+            cmd[pid] = cmd[pid] + " " + arg
+        else:
+            cmd[pid] = arg
+        if pid not in pcomm:
+            pcomm[pid] = task

tools/execsnoop_example.txt

+Demonstrations of execsnoop, the Linux eBPF/bcc version.
+
+
+execsnoop traces new processes. For example:
+
+# ./execsnoop 
+PCOMM            PID    RET ARGS
+supervise        9660     0 ./run
+supervise        9661     0 ./run
+mkdir            9662     0 /bin/mkdir -p ./main
+run              9663     0 ./run
+chown            9664     0 /bin/chown nobody:nobody ./main
+run              9665     0 /bin/mkdir -p ./main
+supervise        9667     0 ./run
+run              9660    -2 /usr/local/bin/setuidgid nobody /command/multilog t ./main
+chown            9668     0 /bin/chown nobody:nobody ./main
+run              9666     0 /bin/chmod 0777 main
+run              9663    -2 /usr/local/bin/setuidgid nobody /command/multilog t ./main
+run              9669     0 /bin/mkdir -p ./main
+run              9661    -2 /usr/local/bin/setuidgid nobody /command/multilog t ./main
+supervise        9670     0 ./run
+[...]
+
+The output shows the parent process/command name (PCOMM), the PID, the return
+value of the exec() (RET), and the filename with arguments (ARGS). The example
+above shows various regular system daemon activity, including some failures
+(trying to execute a /usr/local/bin/setuidgid, which I just noticed doesn't
+exist).
+
+It works by traces the execve() system call (commonly used exec() variant), and
+shows details of the arguments and return value. This catches new processes
+that follow the fork->exec sequence, as well as processes that re-exec()
+themselves. Some applications fork() but do not exec(), eg, for worker
+processes, which won't be included in the execsnoop output.
+
+
+The -X option can be used to only show successful exec()s. For example, tracing
+a "man ls":
+
+# ./execsnoop -X
+PCOMM            PID    RET ARGS
+bash             15887    0 /usr/bin/man ls
+preconv          15894    0 /usr/bin/preconv -e UTF-8
+man              15896    0 /usr/bin/tbl
+man              15897    0 /usr/bin/nroff -mandoc -rLL=169n -rLT=169n -Tutf8
+man              15898    0 /usr/bin/pager -s
+nroff            15900    0 /usr/bin/locale charmap
+nroff            15901    0 /usr/bin/groff -mtty-char -Tutf8 -mandoc -rLL=169n -rLT=169n
+groff            15902    0 /usr/bin/troff -mtty-char -mandoc -rLL=169n -rLT=169n -Tutf8
+groff            15903    0 /usr/bin/grotty
+
+This shows the various commands used to process the "man ls" command.
+
+
+A -t option can be used to include a timestamp column, and a -n option to match
+on a name or substring from the full command line (filename + args). Regular
+expressions are allowed. For example, matching commands containing "mount":
+
+# ./execsnoop -tn mount
+TIME(s) PCOMM            PID    RET ARGS
+2.849   bash             18049    0 /bin/mount -p
+
+
+USAGE message:
+
+# ./execsnoop -h
+usage: execsnoop [-h] [-t] [-X] [-n NAME]
+
+Trace exec() syscalls
+
+optional arguments:
+  -h, --help            show this help message and exit
+  -t, --timestamp       include timestamp on output
+  -X, --excludefails    exclude failed exec()s
+  -n NAME, --name NAME  only print commands matching this name (regex), any
+                        arg
+
+examples:
+    ./execsnoop           # trace all exec() syscalls
+    ./execsnoop -X        # only show successful exec()s
+    ./execsnoop -t        # include timestamps
+    ./execsnoop -n main   # only print command lines containing "main"