As per man netlink, nlmsg_pid is not process id and in fact a value
different from process id can be used.

  bpf: Wrong pid -1615084642, expected 24407

This problem can be triggered by using pyroute2 with bcc.
Signed-off-by: Toshiaki Makita <makita.toshiaki@lab.ntt.co.jp>

2. Modify procfilename buffer length in bcc_procutils_language function.

On Debian Stretch `libfl-dev` is not installed automatically
if you install `flex`, this takjes care of the issue.

permit multiple pids attaching to the same probe

Currently, if more than one pid-associated USDT attaching to
the same probe, usdt readarg code will be generated twice and
the compiler will complain.

This patch solves issue by preventing code duplication if
a previous context with the same mnt point and exec binary
has generated the code for the same probe. The event name is
also changed to have pid embedded so different pid-associated
uprobe event will have different names.

This patch introduces an internal uprobe event name
discrepency. It is a good idea to have event name
generation in libbpf so that both C++ API and Python API
will have consistent name conventions. This will be
addressed in a subsequent commit as it is largely
a different issue.
Signed-off-by: Yonghong Song <yhs@fb.com>

generate proper usdt code to prevent llvm meddling with ctx->#fields

Qin reported a test case where llvm still messes up with ctx->#fields.
For code like below:
  switch(ctx->ip) {
    case 0x7fdf2ede9820ULL: *((int64_t *)dest) = *(volatile int64_t *)&ctx->r12; return 0;
    case 0x7fdf2edecd9cULL: *((int64_t *)dest) = *(volatile int64_t *)&ctx->bx; return 0;
  }
The compiler still generates:
    # r1 is the pointer to the ctx
    r1 += 24
    goto LBB0_4
  LBB0_3:
    r1 += 40
  LBB0_4:
    r3 = *(u64 *)(r1 + 0)
The verifier will reject the above code since the last load is not "ctx + field_offset"
format.

The responsible llvm optimization pass is CFGSimplifyPass. Its main implementation
in llvm/lib/Transforms/Utils/SimplifyCFG.cpp. The main routine to do the optimization
is SinkThenElseCodeToEnd. The routine canSinkInstructions is used to determine whether
an insn is a candidate for sinking.

Unfortunately, volatile load/store is not a condition to prevent the optimization.
But inline assembly is a condition which can prevent further optimization.

In this patch, instead of using volatile to annotate ctx->#field access, we do
normal ctx->#field access but put a compiler inline assembly memory barrier
   __asm__ __volatile__(\"\": : :\"memory\");
after the field access.

Tested with usdt unit test case, usdt_samples example, a couple of usdt unit tests
developed in the past.
Signed-off-by: Yonghong Song <yhs@fb.com>

Support tracing MySQL queries even when MySQL is built
without USDT support, by using uprobes on internal functions
responsible for command (query) dispatching.

Fix a clang memory leak

In clang frontend actions, several compiler invocations are called
for rewriter and transforming source code to IR. During the invocation
to transform source code to IR, CodeGenOpts.DisableFree is used
to control whether the top target machine structure should be
freed or not for a particular clang invocation,
and its default value is TRUE.

See clang:lib/CodeGen/BackendUtil.cpp:
  ~EmitAssemblyHelper() {
    if (CodeGenOpts.DisableFree)
      BuryPointer(std::move(TM));
  }

So by default, the memory held by TM will not freed, even if
BPF module itself is freed. This is even more problematic
when continuous building/loading/unloading happens for long
live service.

This patch explicitly sets CodeGenOpts.DisableFree to FALSE
so memory can be properly freed. I did a simple experiment
to compile/load/unload an empty BPF program and the saving
is roughly 0.5MB.
Signed-off-by: Yonghong Song <yhs@fb.com>

Update bpf.h and virtual_bpf.h to 4.13-rc1

Signed-off-by: Geneviève Bastien <gbastien@versatic.net>

`trace.py` parses a probe using the colon as a separator.  As a result, it
fails to create a uprobe for binary/library with a filename containing colons.

This diff fixes that issue with `trace.py`.  It requires a kernel with
https://lkml.org/lkml/2017/1/13/585 merged to work properly, otherwise
`trace.py` still fails for create uprobes.

* memleak: handle libc allocation functions other than malloc

* memleak: use tracepoints to track kernel allocations

* memleak: add combined-only mode

With large number of outstanding allocations, amount of data passed from
kernel becomes large, which slows everything down.

This patch calculates allocation statistics inside kernel, allowing user-
space part to pull combined statistics data only, thus significantly
reducing amount of passed data.

* memleak: increase hashtable capacities

There are a lot of allocations happen in kernel. Default values are not
enough to keep up.

* test: add a test for the memleak tool

This sample contains:
    - A library with an operation that uses usdt probes.
    - A console application that calls the operation.
    - Scripts to trace the latency of the operation.
    - Corresponding cmake files.

And fix a typo: direct_action -> direct-action

The macro _GNU_SOURCE should be defined to pick up the function
declaration for setns.  Fixes build warning:

"src/cc/libbpf.c:482:7: warning: implicit declaration of
 function 'setns' [-Wimplicit-function-declaration]
   if (setns(target_fd, CLONE_NEWNS)) {"
Signed-off-by: Colin Ian King <colin.king@canonical.com>

Fix bcc.lua build issue in Ubuntu 17.04

Cleanup the `strncmp` code and add a few more ignored map names

In fc25 box, gcc6.3.1 is configured with pie default off.
Here, pie stands for position independent execution.

In ubuntu 17.04, gcc6.3.0, however, is configured with
pie default on. The gcc driver automatically adds -pie
to the linker options.

Since bcc.lua build needs pie off, previously, -fno-pie
is passed to the compiler. -fno-pie is a gcc option
impacting the code generation and it didn't
negate the -pie option in the linker. The correct way
seems to use gcc linker option -no-pie which can
successfully cancel the default -pie in the linker.
Signed-off-by: Yonghong Song <yhs@fb.com>

Treat strings as bytes. This is independent of the larger refactor
(#1139) from which it is cherry-picked)
Signed-off-by: Brenden Blanco <bblanco@gmail.com>

The issue is caused by the following clang change on 5.0:
https://reviews.llvm.org/D28404

Basically, at -O0, unless always inlining is specified, a
function will be marked as optnone and noinline.

This causes two kinds of issues: (1). optnone will generate
suboptimal code with heavy stack use and this high likely can
cause verifier failure; and (2). even if user mark all his/her
defined functions in bpf program as always inlining, some
functions in linux header files are not marked as always inline
and hence will be marked as noinline instead, ultimately
causing llvm complaining about global function reference.

This patch bumps the clang optimization level to -O2.
This should work with older versions of llvm as well.
Signed-off-by: Yonghong Song <yhs@fb.com>

Update the installation instructions for Fedora

Signed-off-by: Yonghong Song <yhs@fb.com>

cnb@ubuntu-14:~/iovisor/bcc/build$ make
[  6%] Built target clang_frontend
[  9%] Built target bpf-static
[ 16%] Built target bcc-loader-static
[ 30%] Built target b_frontend
[ 47%] Built target bcc-static
[ 48%] Built target CPUDistribution
[ 50%] Built target FollyRequestContextSwitch
[ 51%] Built target HelloWorld
[ 52%] Built target LLCStat
[ 54%] Built target RandomRead
[ 55%] Built target RecordMySQLQuery
[ 56%] Built target TCPSendStack
[ 80%] Built target bcc-shared
[ 83%] Built target bpf-shared
[ 84%] Built target bcc_py
Linking C executable bcc-lua
cc: error: unrecognized command line option â-no-pieâ
make[2]: *** [src/lua/bcc-lua] Error 1
make[1]: *** [src/lua/CMakeFiles/bcc-lua.dir/all] Error

option is called -fno-pie

Fixes #782. Solution taken verbatim from @jepio here:

https://github.com/iovisor/bcc/issues/782#issuecomment-259075010

I ran into the same issue attempting to compile from source on
a fresh Ubuntu 16.10/Yakkety host:

Linking C executable bcc-lua
/usr/bin/ld: libluajit-5.1.a(ljamalg.o): relocation R_X86_64_32S
    against `.rodata' can not be used when making a shared object;
    recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output

Build succeeded after patch was applied.

o This is caused by my ignorant error in previous usdt change.
  Obviously, I need to store into pointer address in order to
  propagate value back to caller.
o Thanks Tetsuo Handa pointing this out.
Signed-off-by: Yonghong Song <yhs@fb.com>