python: make _decode_table_types aware of __int128

Do not keep Loader instances around

Fix 'test_libbcc' from failing in certain scenarios

man/bps: Add a man page for introspection/bps.c

There are no ctypes types ready for int128. However, recognizing this
type and representing it as an array is better than simply erroring out.

* Add mapping between (unsigned) __128 C types declaration and ctypes
  parsing
* Add simple test case in test_clang to recognisze and use such types
* Fixes deprecation warnings about self.assertEquals in 2 other test
  cases.

This patch adds a man page for bps.c and also introduces
bps in README.md
Signed-off-by: Martin KaFai Lau <kafai@fb.com>

libbpf: Enable the creation of unbound raw socket

force linking the whole api-static library into shared library

When a static library is linked to produce a shared library,
only referenced symbols in the static library eventually
gets linked.

api-static library has entry points for C++ API.
Many symbols in this library do not have references
outside of this library and will get dropped during
linking process.

This patch forces the linking of the whole api-static
library in order to procude bcc shared library.
Reported-by: Mauricio Vasquez <mauricio.vasquez@polito.it>
Signed-off-by: Yonghong Song <yhs@fb.com>

funcslower: add a note about nested calls

Passing a NULL (or empty) char string to bpf_open_raw_sock() will skip
the bind call on the newly created raw socket. This can create sniffers
for all interfaces using a single socket filter.

The dns_matching example has been updated to default to 'any' interface
if noone is specified in the arguments.

funcslower: fix missing tgid when filtering by pid

* The name encoding function was not checking the constraints on domain
  names properly (<= 253 chars as one byte is needed for the name of the
  label and one for the terminating 0-len label; <= 63 chars per label).
* The name encoding function was erroring when assigning a struct value
  in the byte array (Python 3.6.3). Refactored to join successive
  subarrays, and moved the null padding to make it explicit that it is
  needed by the bpf map key (and not the dns encoding).
* Used builtin from argparse to have a list of domains in the command
  line arguments.
* Reset the non-block flag through fcntl directly instead of
  reconstructing a socket object.
* Exit gracefully when triggering SIGINT as hinted.

Update info about installing stable packages
for Ubuntu Xenial in INSTALL.md

Fixes:#1440

Update perf event type and config checks

solisten.py maintenance

This fixes 'test_usdt_probes', called from 'test_libbcc', from
failing if the static markers are in libruby rather than a ruby
executable. So, when the path of the binary having the markers
is tested, we must check if either "/ruby" or "/libruby" is a
substring rather than only checking for the former.

Also, rather than relying on hardcoded values for the number of
locations and arguments for a given probe, we capture this info
using the readelf tool and then compare these with the values
reported by bcc. This is because these numbers may vary across
different versions of ruby and will cause the tests to fail.
For example, 'test_libbcc' fails on Fedora 26 with ruby-2.4 as
the number of locations where it expects 'array__create' is 7
even when it is actually 2 as reported by bcc and verified with
readelf output.
Signed-off-by: Sandipan Das <sandipan@linux.vnet.ibm.com>

This fixes 'test_c_api', called from 'test_libbcc', from failing
if the name of the first symbol with the address of "strtok" in
libc as returned by bcc_symcache_resolve() is not "strtok" but
some other alias pointing to the same address. For glibc-2.25 on
powerpc, a mismatch occurs as the name of the first symbol with
the same address as that of "strtok" is "__GI_strtok". This can
be verified from the output of the nm tool on Fedora 26 ppc64le
as shown below.

  00000000000b59c0 t __GI_strtok
  00000000000b59c0 T strtok
Signed-off-by: Sandipan Das <sandipan@linux.vnet.ibm.com>

This introduces a separate source file for adding common
utilities which can be used by any of the cc tests. For
now, we add a function to read values from the output of
a command with a format string like scanf().
Signed-off-by: Sandipan Das <sandipan@linux.vnet.ibm.com>

Signed-off-by: Jean-Tiare Le Bigot <jt@yadutaf.fr>

Signed-off-by: Jean-Tiare Le Bigot <jt@yadutaf.fr>

Make 'tools/wakeuptime' use stack maps

This moves the original wakeuptime script that uses
manual stack walking to the old tools directory.
Signed-off-by: Sandipan Das <sandipan@linux.vnet.ibm.com>

This updates the manpage and examples for 'tools/wakeuptime'
so that they include descriptions of the new arguments.
Signed-off-by: Sandipan Das <sandipan@linux.vnet.ibm.com>

This fixes 'test_tools_smoke' from failing on systems
running linux < 4.6 because of the wakeuptime script.
This script uses the bpf stack maps feature which is
available only for linux 4.6+. To avoid this, we skip
running it if the minimum kernel version requirement
is not met.
Signed-off-by: Sandipan Das <sandipan@linux.vnet.ibm.com>

Fix 'test_tools_smoke' from failing due to timeout

This modifies 'tools/wakeuptime' to include a filter
in offcpu() for capturing data only for a given pid
or for user threads only.
Signed-off-by: Sandipan Das <sandipan@linux.vnet.ibm.com>

This fixes 'test_tools_smoke' from failing if ucalls and
vfscount need more time to terminate than the specified
timeout duration when running on certain architectures
such as powerpc. To avoid this, we increase the timeout
duration.
Signed-off-by: Sandipan Das <sandipan@linux.vnet.ibm.com>

This modifies 'tools/wakeuptime' to use the stack maps feature.
Rather than using manual, depth-limited stack walking with some
architecture-specific code, using stack maps makes the script
usable across different architectures. This also adds command
line arguments to let the user to set the minimum and maximum
amount of time in microseconds over which traces are stored.
Signed-off-by: Sandipan Das <sandipan@linux.vnet.ibm.com>

use user-provided log_level in bpf_prog_load

For the case where user provided a log buffer, the
user provided log_level is already been used.

For the case where user provided a log_level and bcc
needs to allocate buffer, the log_level is fixed to 1.
Use user provided log level instead.
Signed-off-by: Yonghong Song <yhs@fb.com>

Move BCC debug options to a installed header file

Kernel Commit 29e48ce87f1e ("task_struct: Allow randomized layout")
(https://patchwork.kernel.org/patch/9797817/)
permits to randomize a section of kernel task_struct data
structure. This only takes effect when CONFIG_GCC_PLUGINS and
CONFIG_GCC_PLUGIN_RANDSTRUCT are set.

When randomization does not happen, an anonymous struct
is introduced in task_struct data structure by gcc when compiling
the kernel. To make field offset compatible, task_struct will
have the same anonymous struct. The patch is at
http://www.spinics.net/lists/kernel/msg2644958.html.
Otherwise, bpf program may get wrong data from the kernel.

Currently, bcc bpf_probe_read rewriter does not support
anonymous struct/union. For example, with the above
compiler-clang.h patch, examples/tracing/task_switch.py
will have the following error:
  /virtual/main.c:16:18: error: internal error: opLoc is invalid while preparing probe rewrite
    key.prev_pid = prev->pid;
                   ^

For anonymous structure, opcode source location ("->") is not available
and hence the above failure. We could use memberLoc ("pid") which is
available for anonymous struct/union. For example, for
  struct sock *skp; ...; u32 saddr = skp->__sk_common.skc_rcv_saddr;
The old way for bpf_probe_read rewrite:
  bpf_probe_read(&_val, sizeof(_val), (u64)skp + offsetof(struct sock, __sk_common.skc_rcv_saddr));
The new way:
  bpf_probe_read(&_val, sizeof(_val), (u64)&skp->__sk_common.skc_rcv_saddr);

The new way is similar to what typical bpf programs may do manually.
Signed-off-by: Yonghong Song <yhs@fb.com>

When BPFModule finalized the module, it set UseOrcMCJITReplacement to
true to use OrcJIT for ExecutionEngine. However, this never worked.
First, in clang_libs.cmake, mcjit is in the library list instead of
orcjit, so ExecutionEngine always fell back to MCJIT.
Second, even if OrcJIT was linked correctly, it actually broke bcc.
For OrcJIT, finalizeObject() is just an empty function. The code
generation is delayed till getPointerToFunction() or runFunction(), so
the current implementation of BPFModule won't work for OrcJIT.

This bug was covered when using the separate LLVM share libraries. If
the system builds LLVM into an unified share library, then OrcJIT will
be used and bcc would fail to generate BPF bytecodes without any
warning.
Signed-off-by: Gary Lin <glin@suse.com>