Commit 5e4e1f46 authored by Sasha Goldshtein's avatar Sasha Goldshtein

Updated man and examples with new probe syntax and command-line switches

parent ed21adfc
...@@ -65,6 +65,7 @@ Examples: ...@@ -65,6 +65,7 @@ Examples:
Tools: Tools:
- tools/[argdist](tools/argdist.py): Display function parameter values as a histogram or frequency count. [Examples](tools/argdist_example.txt).
- tools/[bashreadline](tools/bashreadline.py): Print entered bash commands system wide. [Examples](tools/bashreadline_example.txt). - tools/[bashreadline](tools/bashreadline.py): Print entered bash commands system wide. [Examples](tools/bashreadline_example.txt).
- tools/[biolatency](tools/biolatency.py): Summarize block device I/O latency as a histogram. [Examples](tools/biolatency_example.txt). - tools/[biolatency](tools/biolatency.py): Summarize block device I/O latency as a histogram. [Examples](tools/biolatency_example.txt).
- tools/[biotop](tools/biotop.py): Top for disks: Summarize block device I/O by process. [Examples](tools/biotop_example.txt). - tools/[biotop](tools/biotop.py): Top for disks: Summarize block device I/O by process. [Examples](tools/biotop_example.txt).
...@@ -79,7 +80,6 @@ Tools: ...@@ -79,7 +80,6 @@ Tools:
- tools/[filetop](tools/filetop.py): File reads and writes by filename and process. Top for files. [Examples](tools/filetop_example.txt). - tools/[filetop](tools/filetop.py): File reads and writes by filename and process. Top for files. [Examples](tools/filetop_example.txt).
- tools/[funccount](tools/funccount.py): Count kernel function calls. [Examples](tools/funccount_example.txt). - tools/[funccount](tools/funccount.py): Count kernel function calls. [Examples](tools/funccount_example.txt).
- tools/[funclatency](tools/funclatency.py): Time kernel functions and show their latency distribution. [Examples](tools/funclatency_example.txt). - tools/[funclatency](tools/funclatency.py): Time kernel functions and show their latency distribution. [Examples](tools/funclatency_example.txt).
- tools/[gentrace](tools/gentrace.py): Trace function parameter values as a histogram or raw data. [Examples](tools/gentrace_examples.txt).
- tools/[gethostlatency](tools/gethostlatency.py): Show latency for getaddrinfo/gethostbyname[2] calls. [Examples](tools/gethostlatency_example.txt). - tools/[gethostlatency](tools/gethostlatency.py): Show latency for getaddrinfo/gethostbyname[2] calls. [Examples](tools/gethostlatency_example.txt).
- tools/[hardirqs](tools/hardirqs.py): Measure hard IRQ (hard interrupt) event time. [Examples](tools/hardirqs_example.txt). - tools/[hardirqs](tools/hardirqs.py): Measure hard IRQ (hard interrupt) event time. [Examples](tools/hardirqs_example.txt).
- tools/[killsnoop](tools/killsnoop.py): Trace signals issued by the kill() syscall. [Examples](tools/killsnoop_example.txt). - tools/[killsnoop](tools/killsnoop.py): Trace signals issued by the kill() syscall. [Examples](tools/killsnoop_example.txt).
......
.TH gentrace 8 "2016-02-11" "USER COMMANDS" .TH argdist 8 "2016-02-11" "USER COMMANDS"
.SH NAME .SH NAME
gentrace \- Trace a function and display a histogram or summary of its parameter values. Uses Linux eBPF/bcc. argdist \- Trace a function and display a histogram or frequency count of its parameter values. Uses Linux eBPF/bcc.
.SH SYNOPSIS .SH SYNOPSIS
.B gentrace [-h] [-p PID] [-z STRING_SIZE] [-i INTERVAL] [-c COUNT] specifier [specifier ...] .B argdist [-h] [-p PID] [-z STRING_SIZE] [-i INTERVAL] [-n COUNT] [-H specifier [specifier ...]] [-C specifier [specifier ...]]
.SH DESCRIPTION .SH DESCRIPTION
gentrace attaches to function entry and exit points, collects specified parameter argdist attaches to function entry and exit points, collects specified parameter
values, and stores them in a histogram or a raw counting collection that counts values, and stores them in a histogram or a frequency collection that counts
the number of times a parameter value occurred. It can also filter parameter the number of times a parameter value occurred. It can also filter parameter
values and instrument multiple entry points at once. values and instrument multiple entry points at once.
...@@ -18,8 +18,7 @@ CONFIG_BPF and bcc. ...@@ -18,8 +18,7 @@ CONFIG_BPF and bcc.
Print usage message. Print usage message.
.TP .TP
\-p PID \-p PID
Trace only functions in the process PID. This filter will only apply to user-space Trace only functions in the process PID.
functions. If you only provide kernel-space probes, the filter is ignored.
.TP .TP
\-z STRING_SIZE \-z STRING_SIZE
When collecting string arguments (of type char*), collect up to STRING_SIZE When collecting string arguments (of type char*), collect up to STRING_SIZE
...@@ -28,88 +27,98 @@ characters. Longer strings will be truncated. ...@@ -28,88 +27,98 @@ characters. Longer strings will be truncated.
-i INTERVAL -i INTERVAL
Print the collected data every INTERVAL seconds. The default is 1 second. Print the collected data every INTERVAL seconds. The default is 1 second.
.TP .TP
-c COUNT -n NUMBER
Print the collected data COUNT times and then exit. Print the collected data COUNT times and then exit.
.TP .TP
SPECIFIER -H SPECIFIER, -C SPECIFIER
One or more probe specifications that instruct gentrace which functions to One or more probe specifications that instruct argdist which functions to
probe, which parameters to collect, how to aggregate them, and whether to perform probe, which parameters to collect, how to aggregate them, and whether to perform
any filtering. See SPECIFIER SYNTAX below. any filtering. See SPECIFIER SYNTAX below.
.SH SPECIFIER SYNTAX .SH SPECIFIER SYNTAX
The general specifier syntax is as follows: The general specifier syntax is as follows:
.B <raw|hist>[-ret]:[library]:function(signature)[:type:expr[:filter]] .B {p,r}:[library]:function(signature)[:type:expr[:filter]][;label]
.TP .TP
Probe type \- "raw", "hist", "raw-ret", "hist-ret". .B {p,r}
Indicates where to place the probe and whether the probe should collect raw Probe type \- "p" for function entry, "r" for function return;
event information, or aggregate the collected values into a histogram. Raw \-H for histogram collection, \-C for frequency count.
Indicates where to place the probe and whether the probe should collect frequency
count information, or aggregate the collected values into a histogram. Counting
probes will collect the number of times every parameter value was observed, probes will collect the number of times every parameter value was observed,
whereas histogram probes will collect the parameter values into a histogram. whereas histogram probes will collect the parameter values into a histogram.
Only integral types can be used with histogram probes; there is no such limitation Only integral types can be used with histogram probes; there is no such limitation
for raw probes. Suffix with \-ret to indicate that the probe should be placed for counting probes. Return probes can only use the pseudo-variable $retval, which
at function return. This probe can only use the pseudo-variable @retval, which
is an alias for the function's return value. is an alias for the function's return value.
.TP .TP
.B [library]
Library containing the probe. Library containing the probe.
Specify the full path to the .so or executable file where the function to probe Specify the full path to the .so or executable file where the function to probe
resides. Alternatively, you can specify just the lib name: for example, "c" resides. Alternatively, you can specify just the lib name: for example, "c"
refers to libc. If no library name is specified, the kernel is assumed. refers to libc. If no library name is specified, the kernel is assumed.
.TP .TP
.B function(signature)
The function to probe, and its signature. The function to probe, and its signature.
The function name must match exactly for the probe to be placed. The signature, The function name must match exactly for the probe to be placed. The signature,
on the other hand, is only required if you plan to collect parameter values on the other hand, is only required if you plan to collect parameter values
based on that signature. For example, if you only want to collect the first based on that signature. For example, if you only want to collect the first
parameter, you don't have to specify the rest of the parameters in the signature. parameter, you don't have to specify the rest of the parameters in the signature.
.TP .TP
.B [type]
The type of the expression to capture. The type of the expression to capture.
This is the type of the keys in the histogram or raw event collection that are This is the type of the keys in the histogram or raw event collection that are
collected by the probes. collected by the probes.
.TP .TP
.B [expr]
The expression to capture. The expression to capture.
These are the values that are assigned to the histogram or raw event collection. These are the values that are assigned to the histogram or raw event collection.
You may use the parameters directly, or valid C expressions that involve the You may use the parameters directly, or valid C expressions that involve the
parameters, such as "size % 10". parameters, such as "size % 10".
.TP .TP
A filter applied to the captured data. .B [filter]
The filter applied to the captured data.
Only parameter values that pass the filter will be collected. This is any valid Only parameter values that pass the filter will be collected. This is any valid
C expression that refers to the parameter values, such as "fd == 1 && length > 16". C expression that refers to the parameter values, such as "fd == 1 && length > 16".
.TP
.B [label]
The label that will be displayed when printing the probed values. By default,
this is the probe specifier.
.SH EXAMPLES .SH EXAMPLES
.TP .TP
Print a histogram of allocation sizes passed to kmalloc: Print a histogram of allocation sizes passed to kmalloc:
# #
.B gentrace.py 'hist::__kmalloc(u64 size):u64:size' .B argdist.py -H 'p::__kmalloc(u64 size):u64:size'
.TP .TP
Print a raw count of how many times process 1005 called malloc with an allocation size of 16 bytes: Print a count of how many times process 1005 called malloc with an allocation size of 16 bytes:
# #
.B gentrace.py -p 1005 'raw:c:malloc(size_t size):size_t:size:size==16' .B argdist.py -p 1005 -C 'p:c:malloc(size_t size):size_t:size:size==16'
.TP .TP
Snoop on all strings returned by gets(): Snoop on all strings returned by gets():
# #
.B gentrace.py 'raw-ret:c:gets():char*:@retval' .B argdist.py -C 'r:c:gets():char*:$retval'
.TP .TP
Print raw counts of how many times writes were issued to a particular file descriptor number, in process 1005: Print frequency counts of how many times writes were issued to a particular file descriptor number, in process 1005:
# #
.B gentrace.py -p 1005 'raw:c:write(int fd):int:fd' .B argdist.py -p 1005 -C 'p:c:write(int fd):int:fd'
.TP .TP
Print a histogram of error codes returned by read() in process 1005: Print a histogram of error codes returned by read() in process 1005:
# #
.B gentrace.py -p 1005 'hist-ret:c:read()' .B argdist.py -p 1005 -H 'r:c:read()'
.TP .TP
Print a histogram of buffer sizes passed to write() across all processes, where the file descriptor was 1 (STDOUT): Print a histogram of buffer sizes passed to write() across all processes, where the file descriptor was 1 (STDOUT):
# #
.B gentrace.py 'hist:c:write(int fd, const void *buf, size_t count):size_t:count:fd==1' .B argdist.py -H 'p:c:write(int fd, const void *buf, size_t count):size_t:count:fd==1'
.TP .TP
Count fork() calls in libc across all processes: Count fork() calls in libc across all processes, grouped by pid:
# #
.B gentrace.py 'raw:c:fork' .B argdist.py -C 'p:c:fork():int:$PID;fork per process'
.TP .TP
Print histograms of sleep() and nanosleep() parameter values: Print histograms of sleep() and nanosleep() parameter values:
# #
.B gentrace.py 'hist:c:sleep(u32 seconds):u32:seconds' 'hist:c:nanosleep(struct timespec { time_t tv_sec; long tv_nsec; } *req):long:req->tv_nsec' .B argdist.py -H 'p:c:sleep(u32 seconds):u32:seconds' -H 'p:c:nanosleep(struct timespec { time_t tv_sec; long tv_nsec; } *req):long:req->tv_nsec'
.TP .TP
Spy on writes to STDOUT performed by process 2780, up to a string size of 120 characters: Spy on writes to STDOUT performed by process 2780, up to a string size of 120 characters:
# #
.B gentrace.py -p 2780 -z 120 'raw:c:write(int fd, char* buf, size_t len):char*:buf:fd==1' .B argdist.py -p 2780 -z 120 -C 'p:c:write(int fd, char* buf, size_t len):char*:buf:fd==1'
.SH SOURCE .SH SOURCE
This is from bcc. This is from bcc.
.IP .IP
......
...@@ -28,6 +28,14 @@ int PROBENAME(struct pt_regs *ctx SIGNATURE) ...@@ -28,6 +28,14 @@ int PROBENAME(struct pt_regs *ctx SIGNATURE)
} }
""" """
next_probe_index = 0 next_probe_index = 0
aliases = { "$PID": "bpf_get_current_pid_tgid()" }
def _substitute_aliases(self, expr):
if expr is None:
return expr
for alias, subst in Specifier.aliases.items():
expr = expr.replace(alias, subst)
return expr
def __init__(self, type, specifier, pid): def __init__(self, type, specifier, pid):
self.raw_spec = specifier self.raw_spec = specifier
...@@ -66,6 +74,8 @@ int PROBENAME(struct pt_regs *ctx SIGNATURE) ...@@ -66,6 +74,8 @@ int PROBENAME(struct pt_regs *ctx SIGNATURE)
if self.filter is not None: if self.filter is not None:
self.filter = self.filter.replace("$retval", self.filter = self.filter.replace("$retval",
"(%s)ctx->ax" % self.expr_type) "(%s)ctx->ax" % self.expr_type)
self.expr = self._substitute_aliases(self.expr)
self.filter = self._substitute_aliases(self.filter)
self.pid = pid self.pid = pid
self.probe_func_name = "%s_probe%d" % \ self.probe_func_name = "%s_probe%d" % \
(self.function, Specifier.next_probe_index) (self.function, Specifier.next_probe_index)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment