Refactor external pointer assignments

sync BPF compat headers with latest bpf-next, update BPF features list

Update doc/kernel-versions.md with latest eBPF features, map types,
JIT-compiler, helpers.

Synchronise headers with bpf-next (at commit bcece5dc40b9). Add
prototypes for the following helpers:

- bpf_get_stack()
- bpf_skb_load_bytes_relative()
- bpf_fib_lookup()
- bpf_sock_hash_update()
- bpf_msg_redirect_hash()
- bpf_sk_redirect_hash()
- bpf_lwt_push_encap()
- bpf_lwt_seg6_store_bytes()
- bpf_lwt_seg6_adjust_srh()
- bpf_lwt_seg6_action()
- bpf_rc_repeat()
- bpf_rc_keydown()

The code to track assignments of external pointers was scattered
between VisitVarDecl and VisitBinaryOperator.  This commit defines a
shared assignsExtPtr method.

Doing so also fixes a bug as VisitVarDecl was missing a case for
external pointers retrieved from maps.

Add support for attaching kprobes at custom offsets

tcplife: fix dport filter on tracepoints

let rewriter add code to define CONFIG_CC_STACKPROTECTOR

When using tracepoints, the destination port is retrieved in host
byte order and there is no need to convert it, contrary to the
kprobe version.

Currently, attach_kprobe() only allows kprobes to be attached
to an arch-dependent default location usually in the prologue
of the function corresponding to the event.

With these changes, one can attach a kprobe at a custom offset
from the start of the function.
Signed-off-by: Sandipan Das <sandipan@linux.vnet.ibm.com>

* Added vlan_filter application.

* Added demo application and changed timestamp to human readable format.

* changed files to executable and updated README.md file dependencies part.

* Fixed header printout to match actual output and README.

zfsdist: fix for python3

within tc_perf_event.py,ping command fix

The BPF.get_kprobe_functions method tests if the passed argument
matches with a line of kallsyms, which is opened in binary mode.
Therefore the regex pattern must be bytes as well.

Fix issue #1730

Linux kernel commit 2bc2f688fdf8 ("Makefile: move stack-protector
availability out of Kconfig") moved CONFIG_CC_STACKPROTECTOR
from Kconfig to Makefile. Commit 44c6dc940b19 ("Makefile: introduce
CONFIG_CC_STACKPROTECTOR_AUTO") introduced CONFIG_CC_STACKPROTECTOR_AUTO.

Whether CONFIG_CC_STACKPROTECTOR is defined depends on
CONFIG_CC_STACKPROTECTOR_{AUTO,REGULAR,STRONG}. Since the clang supports
stack-protector, CONFIG_CC_STACKPROTECTOR_AUTO will imply
CONFIG_CC_STACKPROTECTOR for gcc/clang based compilation.

Such changes are introduced in 4.16. For example, the following code
is defined in linux/include/linux/sched.h,
```
        pid_t                           pid;
        pid_t                           tgid;

        /* Canary value for the -fstack-protector GCC feature: */
        unsigned long                   stack_canary;
        /*
         * Pointers to the (original) parent process, youngest child, younger sibling,
         * older sibling, respectively.  (p->father can be replaced with
         * p->real_parent->pid)
         */

        /* Real parent process: */
        struct task_struct __rcu        *real_parent;
```
If kernel config has CONFIG_CC_STACKPROTECTOR_{STRONG,REGULAR,AUTO} defined,
CONFIG_CC_STACKPROTECTOR will be defined in compilation flags by kernel toplevel Makefile.
But since CONFIG_CC_STACKPROTECTOR is not defined in configuration file autoconf.h,
bcc will consider it is not defined. This will cause bcc to access wrong data
in task_struct for any fields after the above stack_canary.

Instead to fix any individual tool, in this patch the bcc rewriter added necessary
macro definition for CONFIG_CC_STACKPROTECTOR in the source code, depending on
CONFIG_CC_STACKPROTECTOR_{AUTO,REGULAR,STRONG}.
Signed-off-by: Yonghong Song <yhs@fb.com>

Fix smoke test for tcplife

fix tcplife.py rewriter issue

Fix dereference replacements for pointers to pointers

rewriter tried to rewrite an argument for a user written
bpf_probe_read and triggers a clang compilation error.

  $ tcplife.py
  /virtual/main.c:134:41: error: cannot take the address of an rvalue of type 'typeof(u64)' (aka 'unsigned long long')
    ...&({ typeof(u64) _val; __builtin_memset(&_val, 0, sizeof(_val)); bpf_probe_read(&_val, sizeof(_val), (u64)&tp->bytes_received); _val; }));
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /virtual/main.c:135:41: error: cannot take the address of an rvalue of type 'typeof(u64)' (aka 'unsigned long long')
    ...&({ typeof(u64) _val; __builtin_memset(&_val, 0, sizeof(_val)); bpf_probe_read(&_val, sizeof(_val), (u64)&tp->bytes_acked); _val; }));
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  2 errors generated.

changing bpf_probe_read to regular pointer access fixed the issue.
Signed-off-by: Yonghong Song <yhs@fb.com>

Currently, the bcc rewriter is unable to track external pointers if
there is more than a single level of indirection (e.g., pointer to
external pointer).  For example, in the following, the rewriter is
unable to detect that ptr2 doesn't need a call to bpf_probe_read,
only *ptr2 do.

int test(struct pt_regs *ctx, struct sock *sk) {
    struct sock *ptr1;
    struct sock **ptr2 = &ptr1;
    *ptr2 = sk;
    return ((struct sock *)(*ptr2))->sk_daddr;
}

This commit fixes this issue by tracking the levels of indirections
in addition to the variable declarations (identifies each variable).
When traversing dereferences, the level of indirections is used to
decide whether the base expression is an external pointer.  The level
of indirections is inherited when a pointer is assigned to a new
variable (assignments and function calls).

execsnoop: don't print newlines in argv

by escaping newlines. Fixes #1037

* Before:
```
$ sudo /usr/share/bcc/tools/execsnoop
PCOMM            PID    PPID   RET ARGS
awk              9910   7831     0 /usr/bin/awk
BEGIN { print "hi" }
```

* With this patch:
```
$ sudo /usr/share/bcc/tools/execsnoop
PCOMM            PID    PPID   RET ARGS
awk              10033  7831     0 /usr/bin/awk \nBEGIN { print "hi" }
```

Limit dereference rewriter to tracing contexts

We should only track and rewrite external pointers from the context
pointer for tracing programs.  Other types of context pointers point
to e.g. packets and do not require a rewrite to a bpf_probe_read
call.

* Add -d (duration) option to argdist, funclatency and syscount

* Add -d option to man pages and _example.txt

usdt: fail when binary doesn't exist. Fixes #1749

And add error message to hint if the problem is that the passed binary
path is not absolute or if the binary doesn't exist.

In case the PID is correct:

* but the binary couldn't be found, it will print:

  ```
  HINT: Specified binary doesn't exist.
  [...]
  ```

* but the binary is not absolute:

  ```
  HINT: Binary path should be absolute.
  [...]
  ```

Otherwise, it should keep behaving as before.

xfsslower: Fix compilation error due to rewriter update

Since ad2d0d9f, the bcc rewriter is able to track more external
pointers going through maps.  xfsslower and zfsslower were relying on
the rewriter not being able to replace some dereferences.  This
commit takes this into account and removes two unnecessary calls to
bpf_probe_read.

Add extra_flag option to bpf_attach_perf_event_raw

The bpf_attach_perf_event_raw API is designed to provide maximum
flexibility for people to use advanced features of Kernel Perf Events
with BPF. Some times specifying flags is neccesary, such as if we want
to use `PERF_FLAG_PID_CGROUP` to profile a container. This commit adds
`extra_flag` option to C and C++ interface

link with bpf-static library for bps

* Add stream debug output for C++ USDT class

This commit adds ability to output USDT class debug message to iostream

* USDT::init() as public function

It would be nice for users be able to call init() and see if the probe
exists / well-formatted before sending them to BPF instance

the issue is reported at #1759.

bps does not need any C++ library functions in bcc.
It only needs libbpf. So link it with bpf-static instead
of bcc-static. This avoids pulling in any C++ module/symbolization/usdt
functions and llvm libraries.

On my local box, the binary size is reduced from ~60MB to 44KB.
Signed-off-by: Yonghong Song <yhs@fb.com>

Currently do calculate the syscall prefix in BPF::init, which requires
loading kallsyms etc. But a lot of times the functionality will not be
used. This commit changes that we only calculate the syscall prefix the
first time we call get_syscall_fnname

Also change to use the KSym class directly for better destruct
production