While changing the stack_id to be signed, I accidentally screwed the
check for an invalid stack_id. Some reason I didn't catch this even in
my tests. This patch fixes the issue (thanks Erick Reyes for reporting).

By the way, one weirdness I see is invalid stack_id is printed as -17
when I print it in python.

When I do bpf_trace_printk, I get these ids:
root@localhost:/# cat /d/tracing/trace_pipe
          <idle>-0     [003] .n.3   942.100225: : sid: 15
          <idle>-0     [002] .n.3   943.140393: : sid: 15
     kworker/3:3-1798  [003] ...3   943.422768: : sid: 6
     kworker/3:3-1798  [003] ...3   943.423419: : sid: 6
     kworker/3:3-1798  [003] ...3   943.423967: : sid: 6
   BootAnimation-650   [003] .n.3   949.840268: : sid: 8
          <idle>-0     [003] .n.3   952.360226: : sid: 15
          <idle>-0     [000] ...3   953.100116: : sid: 11
    Binder:571_3-1469  [000] .n.3   953.513328: : sid: 3
          <idle>-0     [003] .n.3   954.760215: : sid: 15
    Binder:571_3-1469  [000] ...3   955.460271: : sid: 18446744073709551599
          <idle>-0     [003] .n.3   957.420275: : sid: 15
 irq/296-cs35l36-662   [000] ...3   958.422890: : sid: 5
     kworker/1:3-1729  [001] ...3   960.485247: : sid: 18446744073709551599
     kworker/1:3-1729  [001] ...3   960.485888: : sid: 18446744073709551599

As an equivalent, when I do a print of the stack_id from the python
code, I get:
stack_id 15
stack_id 15
stack_id 6
stack_id 6
stack_id 6
stack_id 8
stack_id 15
stack_id 11
stack_id 3
stack_id 15
stack_id -17
stack_id 15
stack_id 5
stack_id -17
stack_id -17

This isn't a big deal since the valid stack_ids match, but still
1.8446744e+19 is -1 in 64-bit speak. So I do find that odd.
Reported-by: Erick Reyes <erickreyes@google.com>
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>

This tool detects code sections in the kernel where IRQs or preemption
are disabled for a user-specified amount of time.

Requires a kernel built with CONFIG_DEBUG_PREEMPT and
CONFIG_PREEMPTIRQ_EVENTS.
Signed-off-by: Joel Fernandes <joel@joelfernandes.org>

 fix socket protocol reading

* Fix external pointer propagation in nested dereferences

and fix the count of indirections for addrof of member dereferences
(&A->b).

In nested dereferences, a dereference of an external pointer may give
a new external pointer.  For example, if A is an external pointer,
then *A and A->b should also be considered as external pointers when
appropriate (e.g., in **A or *(A->b)).

In addition, a member dereference is a dereference, so we need to
count it when counting the number of indirections in ProbeChecker.
If we don't, *(&A->b) won't be rewritten correctly as &A->b will be
considered a pointer to an external pointer.

* Tests for the count of indirections in nested dereferences

Fix issue #1830.

After the rewrite, the code approximately becomes
  &({type _val; bpf_probe_read(&_val, sizeof(_val), &(p->m)); _val)

Firstly the rewriting is really unnecessary, and secondly
the compilation will fail since the addressOf cannot take address
of the rvalue _val.

C standard, however, allows the addressOf operand array subscript
expression, e.g.,
  &({type _val; bpf_probe_read(&_val, sizeof(_val), &(p->m)); _val)[0]

This patch intends to fix the problem by avoiding
the rewriting in the first place of addrressOf simple member expression.
It still permits addressOf the array subscript expression.
Signed-off-by: Yonghong Song <yhs@fb.com>

* Fix nested rewrites dereferences

When the rewriter meets a dereference of a member dereference it
fails to properly rewrite them into calls to bpf_probe_read.  The
reason is that Clang is unable to track the position of rewritten
text, but we can accommodate this by inserting text around the
dereference instead of completely rewriting it.  We are already
doing that for member dereference, but not for simple dereference.

* Test for the rewrite of nested dereferences

* Skip instead of bailing out if MemberExpr is not rewritable

* Recognize context member dereferences despite array accesses

For example, the rewriter should recognize, in the following, that
prev is an external pointer retrieved from the context pointer,
despite the access to the second element of the args array.

struct task_struct *prev = (struct task_struct *)ctx->args[1];

The same could be done for the translation of member dereferences to
bpf_probe_read calls, but that would be a little bit more complex (to
retrieve the correct base) and there's currently no tool that would
benefit from it.

* Test for the recognition of ext ptrs from context array

* tools: remove unnecessary bpf_probe_read calls

5d656bc7 made this calls unnecessary.

add __version__  attribute to bcc module

For the test case in this patch below,
 #define _(P) ({typeof(P) val = 0; bpf_probe_read(&val, sizeof(val), &P); val;})
 int count_tcp(struct pt_regs *ctx, struct sk_buff *skb) {
     return _(TCP_SKB_CB(skb)->tcp_gso_size);
 }

The clang AST will consider the whole `_(TCP_SKB_CB(skb)->tcp_gso_size)`
as a MemberExpr during AST traversal. However, it will consider
the start location of the member expression not rewritable.
Without this patch, we will get an error like below:
    /virtual/main.c:15:44: error: expected ';' after return statement
    return _(TCP_SKB_CB(skb)->tcp_gso_size)); _val; });
Basically, the start of bpf_probe_read() rewritingg failed but
later part succeeded, so the code becomes uncompilable.

Previously, we did not see such issues, but as rewriter got
more smarter this bug is exposed.

This patch fixed the issue by preventing rewriting the whole
expression if the start location for the member expression is
not rewritable.
Signed-off-by: Yonghong Song <yhs@fb.com>

* modify ext4slower.py for 4.10+ version

* modify ext4slower.py for 4.10+ version

* #1818

* Trace external pointers through function returns

Surprisingly, the rewriter wasn't able to trace external pointers
returned by inlined functions until now.  This commit fixes it by
adding functions that return an external pointer to ProbeVisitor's
set of external pointers, along with the levels of indirection.

This change requires reversing a few traversals to visit called
functions before they are called.  Then, we check the presence of an
external pointer on return statements and retrieve that information
at the call expression.

* Tests dereferences of ext ptrs returned by inlined func

* tcpdrop: remove unnecessary bpf_probe_read calls

e783567a makes these calls unnecessary.

* Skip all dereferences inside bpf_probe_read calls

If the user decides to rely on a manual call to bpf_probe_read, we
don't try to rewrite its last argument.  This is needed as the
rewriter starts to recognize and rewrite more and more dereferences.

* tools: fix dereferences following 1a765a17

Signed-off-by: Brenden Blanco <bblanco@gmail.com>

Make the input string of get_kprobe_functions as
bytes literal in tcpdrop and zfsslower so the
tool can be python3 compatible.
Signed-off-by: Yonghong Song <yhs@fb.com>

Fix issue #1802.

On x64, the following commit (in 4.17) changed the raw parameter passed to
the syscall entry function from a list of parameters supplied in user space
to a single `pt_regs *` parameter. Also in 4.17, x64 syscall entry function
is changed from `sys_<name>` to `__x64_sys_<name>`.

```
commit fa697140f9a20119a9ec8fd7460cc4314fbdaff3
Author: Dominik Brodowski <linux@dominikbrodowski.net>
Date:   Thu Apr 5 11:53:02 2018 +0200

    syscalls/x86: Use 'struct pt_regs' based syscall calling convention for 64-bit syscalls

    Let's make use of ARCH_HAS_SYSCALL_WRAPPER=y on pure 64-bit x86-64 systems:

    Each syscall defines a stub which takes struct pt_regs as its only
    argument. It decodes just those parameters it needs, e.g:

            asmlinkage long sys_xyzzy(const struct pt_regs *regs)
            {
                    return SyS_xyzzy(regs->di, regs->si, regs->dx);
            }

    This approach avoids leaking random user-provided register content down
    the call chain.

    ...
```

In bcc, we support kprobe function signatures in the bpf program.
The rewriter will automatically generate proper assignment to
these parameters. With the above function signature change, the
original method does not work any more.

This patch enhanced rewriter to generate two version codes guarded
with CONFIG_ARCH_HAS_SYSCALL_WRAPPER. But we need to identify
whether a function will be attached to syscall entry function
or not during prog load time at which time the program has not
attached to any event.

The prefix `kprobe__` is used for kprobe autoload, we can use
`kprobe____x64_sys_` as the prefix to identify x64 syscall entry
functions. To support other architecture or not-autoloading program,
the prefix `syscall__` is introduced to signal it is a syscall
entry function.

trace.py and other tools which uses kprobe syscall entry functions
are also modified with the new interface so that they can
work properly with 4.17.
Signed-off-by: Yonghong Song <yhs@fb.com>

20fb64cd stops the whole AST traversal if it meets a bpf_probe_read call.  I
think the original intent was to simply not rewrite the third argument, so this
commit fixes it by remembering the third argument on bpf_probe_read call
traversals and overriding TraverseStmt to skip the traversal of that argument
when we meet it later.

bcc uses some func prefixes for auto load purpose. These
func prefixes include "kprobe__", "tracepoint__" and
"raw_tracepoint__". Currently we also pass this
function name as the program name to the kernel.

The kernel can only accept 16 bytes so long program
name will be truncated. For example, with bps we will see
something like
     287- <raw_tracepoint>       0      2 Jun10/17:07  raw_tracepoint_
     290- tracepoint             0      4 Jun10/17:08  tracepoint__soc
     297- kprobe                 0      2 Jun10/17:09  kprobe__tcp_cle

Such long prefixes are unnecessarily taking the space
for the real function name. This patch removed such prefixes
before giving them to the kernel.
The result will like below:
     311- <raw_tracepoint>       0      2 Jun10/17:44  sched_switch
     321- tracepoint             0      4 Jun10/17:45  sock__inet_sock
     322- kprobe                 0      2 Jun10/17:45  tcp_cleanup_rbu
Signed-off-by: Yonghong Song <yhs@fb.com>

commit 95b3d8c8 fixed the dport filtering of the kprobes variant by
moving the network byte order to host byte order conversation before the
filtering. Before submitting the perf event the byte order of the dport
was again converted - this commit removes this double conversion.

[V2] Add two map types for bpf_redirect_map()

Smoke test for tcpdrop

adjust tracepoint field type based on size

add missing types in bps

Also add a simple example for DEVMAP based on xdp_drop_count.py

v2:
  Add an example for CPUMAP
Signed-off-by: Gary Lin <glin@suse.com>

Those two map types are necessary to support bpf_redirect_map() in XDP.

v2:
  Use ArrayBase as the base class of DevMap and CpuMap
Signed-off-by: Gary Lin <glin@suse.com>

Add missing program and map types in bps
Signed-off-by: Yonghong Song <yhs@fb.com>

Fix issue #1807

tracepoint may have a format like this:
(from syscalls/sys_enter_socket)
	field:unsigned short common_type;	offset:0;	size:2;	signed:0;
	field:unsigned char common_flags;	offset:2;	size:1;	signed:0;
	field:unsigned char common_preempt_count;	offset:3;	size:1;	signed:0;
	field:int common_pid;	offset:4;	size:4;	signed:1;

	field:int __syscall_nr;	offset:8;	size:4;	signed:1;
	field:int family;	offset:16;	size:8;	signed:0;
	field:int type;	offset:24;	size:8;	signed:0;
	field:int protocol;	offset:32;	size:8;	signed:0;

Current rewriter generates:
    struct tracepoint__syscalls__sys_enter_socket {
	    u64 __do_not_use__;
	    int __syscall_nr;
	    int family;
	    int type;
	    int protocol;
    };

This is incorrect as in the above structure, offsets of
`family`/`type`/`procotol` becomingg 12/16/20.

This patch fixed the issue by adjusting field type based on its size.
The new structure:
    struct tracepoint__syscalls__sys_enter_socket {
	    u64 __do_not_use__;
	    int __syscall_nr;
	    s64 family;
	    s64 type;
	    s64 protocol;
    };
The offsets of all fields are correct now.
Signed-off-by: Yonghong Song <yhs@fb.com>

add tcpdrop tool

Feat/add stack frames to funcslower

profile: remove remnant from old version

The comment on the instrumentation of perf_event_open should have
been removed in commit 715f7e6e.

Signed-off-by: Gary Lin <glin@suse.com>

2. Not using BPF_F_REUSE_STACKID in funcslower stack traces
3. Combining an ifdef check

fix a memory leak for getline()