bpf_probe_read() is often used to access pointees in bpf programs.
Recent rewriter has become smarter so a lot of bpf_probe_read()
can be replaced with simple pointer/member access.

In certain cases, bpf_probe_read() is still preferred though.
For example, kernel net/tcp.h defined TCP_SKB_CB as below
  #define TCP_SKB_CB(__skb)	((struct tcp_skb_cb *)&((__skb)->cb[0]))
User can use below to access tcp_gso_size of a skb data structure.
  TCP_SKB_CB(skb)->tcp_gso_size
The rewriter will fail as it attempts to rewrite (__skb)->cb[0].

Instead of chasing down to prevent exactly the above pattern,
this patch detects function bpf_probe_read() in ProbeVisitor and
will skip it so bpf_probe_read()'s third parameter is a AddrOf.
This can also help other cases where rewriter is not
capable and user used bpf_probe_read() as the workaround.

Also fixed tcptop.py to use direct assignment instead of
bpf_probe_read. Otherwise, rewriter will actually rewrite
src address reference inside the bpf_probe_read().
Signed-off-by: Yonghong Song <yhs@fb.com>

Add "-D __BPF_TRACING__" to frontend compilation flags

In 4.17 kernel, x86 build requires compiler asm-goto support. clang
does not support asm-goto and bpf program compilation started to break.
The following kernel commit

  commit b1ae32dbab50ed19cfc16d225b0fb0114fb13025
  Author: Alexei Starovoitov <ast@kernel.org>
  Date:   Sun May 13 12:32:22 2018 -0700

      x86/cpufeature: Guard asm_volatile_goto usage for BPF compilation

      Workaround for the sake of BPF compilation which utilizes kernel
      headers, but clang does not support ASM GOTO and fails the build.

workarounded the issue by permitting native clang compilation.
A warning message, however, is issued:

  ./arch/x86/include/asm/cpufeature.h:150:2: warning: "Compiler lacks ASM_GOTO support.
        Add -D __BPF_TRACING__ to your compiler arguments" [-W#warnings]
  #warning "Compiler lacks ASM_GOTO support. Add -D __BPF_TRACING__ to your compil...
   ^
  1 warning generated.

This patch added "-D __BPF_TRACING__" to clang frontend compilation to
suppress the warning.
Signed-off-by: Yonghong Song <yhs@fb.com>

Refactor external pointer assignments

sync BPF compat headers with latest bpf-next, update BPF features list

Update doc/kernel-versions.md with latest eBPF features, map types,
JIT-compiler, helpers.

Synchronise headers with bpf-next (at commit bcece5dc40b9). Add
prototypes for the following helpers:

- bpf_get_stack()
- bpf_skb_load_bytes_relative()
- bpf_fib_lookup()
- bpf_sock_hash_update()
- bpf_msg_redirect_hash()
- bpf_sk_redirect_hash()
- bpf_lwt_push_encap()
- bpf_lwt_seg6_store_bytes()
- bpf_lwt_seg6_adjust_srh()
- bpf_lwt_seg6_action()
- bpf_rc_repeat()
- bpf_rc_keydown()

The code to track assignments of external pointers was scattered
between VisitVarDecl and VisitBinaryOperator.  This commit defines a
shared assignsExtPtr method.

Doing so also fixes a bug as VisitVarDecl was missing a case for
external pointers retrieved from maps.

Add support for attaching kprobes at custom offsets

tcplife: fix dport filter on tracepoints

let rewriter add code to define CONFIG_CC_STACKPROTECTOR

When using tracepoints, the destination port is retrieved in host
byte order and there is no need to convert it, contrary to the
kprobe version.

Currently, attach_kprobe() only allows kprobes to be attached
to an arch-dependent default location usually in the prologue
of the function corresponding to the event.

With these changes, one can attach a kprobe at a custom offset
from the start of the function.
Signed-off-by: Sandipan Das <sandipan@linux.vnet.ibm.com>

* Added vlan_filter application.

* Added demo application and changed timestamp to human readable format.

* changed files to executable and updated README.md file dependencies part.

* Fixed header printout to match actual output and README.

zfsdist: fix for python3

within tc_perf_event.py,ping command fix

The BPF.get_kprobe_functions method tests if the passed argument
matches with a line of kallsyms, which is opened in binary mode.
Therefore the regex pattern must be bytes as well.

Fix issue #1730

Linux kernel commit 2bc2f688fdf8 ("Makefile: move stack-protector
availability out of Kconfig") moved CONFIG_CC_STACKPROTECTOR
from Kconfig to Makefile. Commit 44c6dc940b19 ("Makefile: introduce
CONFIG_CC_STACKPROTECTOR_AUTO") introduced CONFIG_CC_STACKPROTECTOR_AUTO.

Whether CONFIG_CC_STACKPROTECTOR is defined depends on
CONFIG_CC_STACKPROTECTOR_{AUTO,REGULAR,STRONG}. Since the clang supports
stack-protector, CONFIG_CC_STACKPROTECTOR_AUTO will imply
CONFIG_CC_STACKPROTECTOR for gcc/clang based compilation.

Such changes are introduced in 4.16. For example, the following code
is defined in linux/include/linux/sched.h,
```
        pid_t                           pid;
        pid_t                           tgid;

        /* Canary value for the -fstack-protector GCC feature: */
        unsigned long                   stack_canary;
        /*
         * Pointers to the (original) parent process, youngest child, younger sibling,
         * older sibling, respectively.  (p->father can be replaced with
         * p->real_parent->pid)
         */

        /* Real parent process: */
        struct task_struct __rcu        *real_parent;
```
If kernel config has CONFIG_CC_STACKPROTECTOR_{STRONG,REGULAR,AUTO} defined,
CONFIG_CC_STACKPROTECTOR will be defined in compilation flags by kernel toplevel Makefile.
But since CONFIG_CC_STACKPROTECTOR is not defined in configuration file autoconf.h,
bcc will consider it is not defined. This will cause bcc to access wrong data
in task_struct for any fields after the above stack_canary.

Instead to fix any individual tool, in this patch the bcc rewriter added necessary
macro definition for CONFIG_CC_STACKPROTECTOR in the source code, depending on
CONFIG_CC_STACKPROTECTOR_{AUTO,REGULAR,STRONG}.
Signed-off-by: Yonghong Song <yhs@fb.com>

Fix smoke test for tcplife

fix tcplife.py rewriter issue

Fix dereference replacements for pointers to pointers

rewriter tried to rewrite an argument for a user written
bpf_probe_read and triggers a clang compilation error.

  $ tcplife.py
  /virtual/main.c:134:41: error: cannot take the address of an rvalue of type 'typeof(u64)' (aka 'unsigned long long')
    ...&({ typeof(u64) _val; __builtin_memset(&_val, 0, sizeof(_val)); bpf_probe_read(&_val, sizeof(_val), (u64)&tp->bytes_received); _val; }));
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /virtual/main.c:135:41: error: cannot take the address of an rvalue of type 'typeof(u64)' (aka 'unsigned long long')
    ...&({ typeof(u64) _val; __builtin_memset(&_val, 0, sizeof(_val)); bpf_probe_read(&_val, sizeof(_val), (u64)&tp->bytes_acked); _val; }));
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  2 errors generated.

changing bpf_probe_read to regular pointer access fixed the issue.
Signed-off-by: Yonghong Song <yhs@fb.com>

Currently, the bcc rewriter is unable to track external pointers if
there is more than a single level of indirection (e.g., pointer to
external pointer).  For example, in the following, the rewriter is
unable to detect that ptr2 doesn't need a call to bpf_probe_read,
only *ptr2 do.

int test(struct pt_regs *ctx, struct sock *sk) {
    struct sock *ptr1;
    struct sock **ptr2 = &ptr1;
    *ptr2 = sk;
    return ((struct sock *)(*ptr2))->sk_daddr;
}

This commit fixes this issue by tracking the levels of indirections
in addition to the variable declarations (identifies each variable).
When traversing dereferences, the level of indirections is used to
decide whether the base expression is an external pointer.  The level
of indirections is inherited when a pointer is assigned to a new
variable (assignments and function calls).

execsnoop: don't print newlines in argv

by escaping newlines. Fixes #1037

* Before:
```
$ sudo /usr/share/bcc/tools/execsnoop
PCOMM            PID    PPID   RET ARGS
awk              9910   7831     0 /usr/bin/awk
BEGIN { print "hi" }
```

* With this patch:
```
$ sudo /usr/share/bcc/tools/execsnoop
PCOMM            PID    PPID   RET ARGS
awk              10033  7831     0 /usr/bin/awk \nBEGIN { print "hi" }
```

Limit dereference rewriter to tracing contexts

We should only track and rewrite external pointers from the context
pointer for tracing programs.  Other types of context pointers point
to e.g. packets and do not require a rewrite to a bpf_probe_read
call.

* Add -d (duration) option to argdist, funclatency and syscount

* Add -d option to man pages and _example.txt

usdt: fail when binary doesn't exist. Fixes #1749

And add error message to hint if the problem is that the passed binary
path is not absolute or if the binary doesn't exist.

In case the PID is correct:

* but the binary couldn't be found, it will print:

  ```
  HINT: Specified binary doesn't exist.
  [...]
  ```

* but the binary is not absolute:

  ```
  HINT: Binary path should be absolute.
  [...]
  ```

Otherwise, it should keep behaving as before.

xfsslower: Fix compilation error due to rewriter update

Since ad2d0d9f, the bcc rewriter is able to track more external
pointers going through maps.  xfsslower and zfsslower were relying on
the rewriter not being able to replace some dereferences.  This
commit takes this into account and removes two unnecessary calls to
bpf_probe_read.

Add extra_flag option to bpf_attach_perf_event_raw

The bpf_attach_perf_event_raw API is designed to provide maximum
flexibility for people to use advanced features of Kernel Perf Events
with BPF. Some times specifying flags is neccesary, such as if we want
to use `PERF_FLAG_PID_CGROUP` to profile a container. This commit adds
`extra_flag` option to C and C++ interface

link with bpf-static library for bps

* Add stream debug output for C++ USDT class

This commit adds ability to output USDT class debug message to iostream

* USDT::init() as public function

It would be nice for users be able to call init() and see if the probe
exists / well-formatted before sending them to BPF instance

the issue is reported at #1759.

bps does not need any C++ library functions in bcc.
It only needs libbpf. So link it with bpf-static instead
of bcc-static. This avoids pulling in any C++ module/symbolization/usdt
functions and llvm libraries.

On my local box, the binary size is reduced from ~60MB to 44KB.
Signed-off-by: Yonghong Song <yhs@fb.com>