When using tracepoints, the destination port is retrieved in host
byte order and there is no need to convert it, contrary to the
kprobe version.

* Added vlan_filter application.

* Added demo application and changed timestamp to human readable format.

* changed files to executable and updated README.md file dependencies part.

* Fixed header printout to match actual output and README.

zfsdist: fix for python3

within tc_perf_event.py,ping command fix

The BPF.get_kprobe_functions method tests if the passed argument
matches with a line of kallsyms, which is opened in binary mode.
Therefore the regex pattern must be bytes as well.

Fix smoke test for tcplife

fix tcplife.py rewriter issue

Fix dereference replacements for pointers to pointers

rewriter tried to rewrite an argument for a user written
bpf_probe_read and triggers a clang compilation error.

  $ tcplife.py
  /virtual/main.c:134:41: error: cannot take the address of an rvalue of type 'typeof(u64)' (aka 'unsigned long long')
    ...&({ typeof(u64) _val; __builtin_memset(&_val, 0, sizeof(_val)); bpf_probe_read(&_val, sizeof(_val), (u64)&tp->bytes_received); _val; }));
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /virtual/main.c:135:41: error: cannot take the address of an rvalue of type 'typeof(u64)' (aka 'unsigned long long')
    ...&({ typeof(u64) _val; __builtin_memset(&_val, 0, sizeof(_val)); bpf_probe_read(&_val, sizeof(_val), (u64)&tp->bytes_acked); _val; }));
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  2 errors generated.

changing bpf_probe_read to regular pointer access fixed the issue.
Signed-off-by: Yonghong Song <yhs@fb.com>

Currently, the bcc rewriter is unable to track external pointers if
there is more than a single level of indirection (e.g., pointer to
external pointer).  For example, in the following, the rewriter is
unable to detect that ptr2 doesn't need a call to bpf_probe_read,
only *ptr2 do.

int test(struct pt_regs *ctx, struct sock *sk) {
    struct sock *ptr1;
    struct sock **ptr2 = &ptr1;
    *ptr2 = sk;
    return ((struct sock *)(*ptr2))->sk_daddr;
}

This commit fixes this issue by tracking the levels of indirections
in addition to the variable declarations (identifies each variable).
When traversing dereferences, the level of indirections is used to
decide whether the base expression is an external pointer.  The level
of indirections is inherited when a pointer is assigned to a new
variable (assignments and function calls).

execsnoop: don't print newlines in argv

by escaping newlines. Fixes #1037

* Before:
```
$ sudo /usr/share/bcc/tools/execsnoop
PCOMM            PID    PPID   RET ARGS
awk              9910   7831     0 /usr/bin/awk
BEGIN { print "hi" }
```

* With this patch:
```
$ sudo /usr/share/bcc/tools/execsnoop
PCOMM            PID    PPID   RET ARGS
awk              10033  7831     0 /usr/bin/awk \nBEGIN { print "hi" }
```

Limit dereference rewriter to tracing contexts

We should only track and rewrite external pointers from the context
pointer for tracing programs.  Other types of context pointers point
to e.g. packets and do not require a rewrite to a bpf_probe_read
call.

* Add -d (duration) option to argdist, funclatency and syscount

* Add -d option to man pages and _example.txt

usdt: fail when binary doesn't exist. Fixes #1749

And add error message to hint if the problem is that the passed binary
path is not absolute or if the binary doesn't exist.

In case the PID is correct:

* but the binary couldn't be found, it will print:

  ```
  HINT: Specified binary doesn't exist.
  [...]
  ```

* but the binary is not absolute:

  ```
  HINT: Binary path should be absolute.
  [...]
  ```

Otherwise, it should keep behaving as before.

xfsslower: Fix compilation error due to rewriter update

Since ad2d0d9f, the bcc rewriter is able to track more external
pointers going through maps.  xfsslower and zfsslower were relying on
the rewriter not being able to replace some dereferences.  This
commit takes this into account and removes two unnecessary calls to
bpf_probe_read.

Add extra_flag option to bpf_attach_perf_event_raw

The bpf_attach_perf_event_raw API is designed to provide maximum
flexibility for people to use advanced features of Kernel Perf Events
with BPF. Some times specifying flags is neccesary, such as if we want
to use `PERF_FLAG_PID_CGROUP` to profile a container. This commit adds
`extra_flag` option to C and C++ interface

link with bpf-static library for bps

* Add stream debug output for C++ USDT class

This commit adds ability to output USDT class debug message to iostream

* USDT::init() as public function

It would be nice for users be able to call init() and see if the probe
exists / well-formatted before sending them to BPF instance

the issue is reported at #1759.

bps does not need any C++ library functions in bcc.
It only needs libbpf. So link it with bpf-static instead
of bcc-static. This avoids pulling in any C++ module/symbolization/usdt
functions and llvm libraries.

On my local box, the binary size is reduced from ~60MB to 44KB.
Signed-off-by: Yonghong Song <yhs@fb.com>

Currently do calculate the syscall prefix in BPF::init, which requires
loading kallsyms etc. But a lot of times the functionality will not be
used. This commit changes that we only calculate the syscall prefix the
first time we call get_syscall_fnname

Also change to use the KSym class directly for better destruct
production

fix get_kprobe_functions

and add `from __future__ import print_function` where needed for Python3
print semantics in Python2

Fix issue #1747.

In commit #1647, we excluded all symbols outside [_stext, _etext].
This is incorrect as it excluded module symbols as well.

This patch changed the algorithm to only skip symbols
in init sections [__init_begin, __init_end].
Signed-off-by: Yonghong Song <yhs@fb.com>

- Fix a crash in the python binding when trying to open a perf buffer
    in python < 3.6
- See https://github.com/iovisor/bcc/issues/1744

This PR fixes the encoding issue in Python3 by converting the argument passed to `find` to a bytearray.

Run queue latency does not make much sense for idle `swapper` threads.

The same happens in `perf sched`:

* https://github.com/torvalds/linux/blob/v4.14/tools/perf/builtin-sched.c

```c
    /*
     * Ignore idle threads:
     */
    if (!strcmp(thread__comm_str(work_list->thread), "swapper"))
        return;
```

```c
static bool is_idle_sample(struct perf_sample *sample,
	       struct perf_evsel *evsel)
{
    /* pid 0 == swapper == idle task */
    if (strcmp(perf_evsel__name(evsel), "sched:sched_switch") == 0)
        return perf_evsel__intval(evsel, sample, "prev_pid") == 0;

    return sample->pid == 0;
}
```

* Add runqslower tool

* Remove mentions of obsolete enqueue_task_* in tools/runq*

* Use u32 for pid field in runqslower

* Trace all external pointers going through a first map

Currently, MapVisitor only detects maps with external pointers as
values if the value was directly passed from a function's argument.
For example, in the following, the rewriter is currently unable to
detect currsock has an external pointer as value because an
intermediate variable is used instead of passing directly sk as the
map's value.

    int test(struct pt_regs *ctx, struct sock *sk) {
        u32 pid = bpf_get_current_pid_tgid();
        struct sock **skp = &sk;
        currsock.update(&pid, skp);
        return 0;
    };

With this commit, MapVisitor is able to trace any external pointer
derived from the function's argument and used as a map value. This
commit breaks the ProbeVisitor traversal in two distinct traversals.
The first rewrites dereferences of external pointers originating
from function's arguments and helpers, while the second rewrites only
dereferences of external pointers passed through maps.
Maps with external pointers as values are identified between the two
ProbeVisitor traversals.

* New tests for external pointers passed through maps

test_ext_ptr_maps_reverse ensures dereferences are correctly replaced
even if the update happens after the lookup (in the order of
MapVisitor traversal).
test_ext_ptr_maps_indirect ensures the rewriter is able to trace
external pointers used as map values even if using an intermediate
variable.

* Fix USDT probes arguments' encoding in Python3

Running `trace` on a binary's USDT while fetching some arguments (
`sudo python3 trace.py -p $(pidof ruby) 'u:ruby:array__create "%d",
arg1'`) fails with `argument 2: <class 'TypeError'>: wrong type`.

This PR fixes the encoding of the USDT probe name in
udst.py `get_probe_arg_ctype` function. I've tested this works on Python 2 too.

Link to article on how Circonus uses bcc