* master: (609 commits)
  docs: references_guide.md: add/fix search examples/tools links (#2186)
  Fix misc file permissions (#2185)
  sync with latest bpf (#2184)
  sync with latest libbpf repo (#2183)
  docs: fix broken link of bpf_log2l(#2176)
  examples/tracing: some minor fixes
  Fix tools/syscount -l (#2180)
  examples/tracing/bitehist.py: add example of linear histogram (#2177)
  cachestat: bring back HITRATIO column
  Fix debuginfo search on Ubuntu
  Add installation instructions for Amazon Linux 1 AMI Sign-Off-By Travis Davies <trdavies@amazon.com>
  [iovisor/bcc] trace: Incorrect symbol offsets when using build_id (#2161) (#2162)
  profile: exclude CPU idle stacks by default (#2166)
  fix cpuunclaimed.py with cfs_rq structure change (#2164)
  tools: rename "deadlock_detector" to "deadlock" (#2152) (#2160)
  use libbpf api in bpf_attach_xdp (#2158)
  support symbol resolution of short-lived process.  (#2144)
  profile.py: return kernel annotations for folded stacks
  use libbpf APIs from libbpf.c (#2156)
  ddos_detector.py to monitor DDOS attacks (#2140)
  ...

Need to include headers for `struct irq_desc` and `struct irq_common_data` that
irq_desc uses without appropriate include.

kirr@deco:~/src/tools/bpf/bcc/tools$ ./pinglat.py
/virtual/main.c:33:31: error: incomplete definition of type 'struct irq_desc'
    const char *irqname = desc->action->name;
                          ~~~~^
include/linux/irqnr.h:9:15: note: forward declaration of 'struct irq_desc'
extern struct irq_desc *irq_to_desc(unsigned int irq);
              ^
1 error generated.
Traceback (most recent call last):
  File "./pinglat.py", line 129, in <module>
    b = BPF(text=prog)
  File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 320, in __init__
    raise Exception("Failed to compile BPF text")
Exception: Failed to compile BPF text

add/fix search examples/tools links in references_guide.md

A few files that are obviously not executable have the exec bit set.
Most those files were created with faulty mode to begin with, but
src/cc/export/helpers.h and src/cc/frontends/clang/loader.cc have had
their exec bit (unintendedly I assume) set by commit bfecc243
("clang: loader: Allow user to override kernel version (#1895)").

The previous sync had a bug in libbpf libbpf_print() function.
This sync brought in the bug fix
Signed-off-by: Yonghong Song <yhs@fb.com>

the libbpf is just sync'ed with latest bpf-next.
Signed-off-by: Yonghong Song <yhs@fb.com>

examples/tracing: some minor fixes

cachestat: bring back HITRATIO column

docs: fix broken link of bpf_log2l()

- chmod +x dddos.py mallocstacks.py stack_buildid_example.py
- Handle Ctrl-C keyboard interrupt for dddos.py, disksnoop.py,
  hello_perf_output.py, stacksnoop.py and trace_perf_output.py

In #2063, syscount's syscall mapping was moved into its own module.
Unfortunately, that broke the "print list of recognized syscalls and
exit" usage of syscount. Fix it by importing the syscall mapping from
the new module.

Add an example of linear histogram, which might be useful to newcomers.
And also update the comments.

Fix debuginfo search on Ubuntu

bcc on Ubuntu picks up the glibc binary instead of the debuginfo file
because find_debug_via_debuglink() doesn't handle symbolic or hard
links when checking if two paths point to the same file.

Add a helper, same_file() which uses the device and inode to check if
the two paths do point to the same file.
Signed-off-by: Anton Blanchard <anton@ozlabs.org>

Sign-Off-By Travis Davies <trdavies@amazon.com>

Fix #2161 - bcc_bsymcache API returned absolute address instead of 
offset from start of the symbol

profile: exclude CPU idle stacks by default

Similar to runqlen.py, make proper adjustment for
cfs_rq_partial structure so it can align with
what the kernel expects.
Signed-off-by: Yonghong Song <yhs@fb.com>

This renames the `deadlock_detector.py` tool to `deadlock.py` to make
the name more diagram-friendly and to be consistent with the naming of the
other tools.

Use libbpf api bpf_set_link_xdp_fd() to attach xdp program to an
interface.

New command line options have been added to tools/trace.py to support the 
new BUILD_ID stackmap. List of symbol files can be added to the script to 
resolve symbols from build id as reported by the kernel in the stack trace
Updated man page and added an example usage

profile.py: return kernel annotations for folded stacks

Recently kernel libbpf was imported in bcc. It makes some of the
libbcc code redundant. This patch modifies libbcc functions to use
libbpf APIs.

 ddos_detector.py to monitor DDOS attacks

Address issue 2154.

When a struct S is used as key to a BPF_HISTOGRAM, it is assumed that the second
member of S holds the slot. But when S is converted to python from bpf C,
a padding may be inserted as a second member. This breaks print_log2_hist().

    root@debian:~/bcc/tools# ./softirqs.py -d
    Tracing soft irq event time... Hit Ctrl-C to end.
    ^C
    Traceback (most recent call last):
      File "./softirqs.py", line 144, in <module>
        dist.print_log2_hist(label, "softirq", section_print_fn=vec_to_name)
      File "/usr/local/lib/python2.7/dist-packages/bcc/table.py", line 326, in print_log2_hist
        vals[slot] = v.value
    TypeError: list indices must be integers, not str

Fix it by skipping the possible padding. Future work would be fixing/working
around in the library where the padding is introduced.

use kernel libbpf in bcc

highlighting the <80 chars, and also adding a note about tool names.

This patch demonstrated how to use libbpf function calls.
Specially, the bcc_map_create is changed to use libbpf
functions bpf_create_map_name and bpf_create_map.
Signed-off-by: Yonghong Song <yhs@fb.com>

The libbpf/src/*.c files are included in the build system,
so those functions will be available for bcc internals to use them.

There are two name conflicts, bpf_create_map and bpf_prog_load,
between src/cc/libbpf.c and src/cc/libbpf/src/{bpf.c,libbpf.c}.
To keep src/cc/libbpf intact, the following renaming happened
in bcc repo:
  bpf_create_map  =>  bcc_create_map
  bpf_prog_load   =>  bcc_prog_load
Signed-off-by: Yonghong Song <yhs@fb.com>

The uapi header
  src/cc/libbpf/include/uapi/linux/bpf.h
is used instead of the current way
  src/cc/compat/linux/bpf.h
Signed-off-by: Yonghong Song <yhs@fb.com>

If the target directory src/cc/libbpf/ does not exist,
the top level CMakeLists.txt is changed to fetch
submodules with the following command
  git submodule update --init --recursive

The dev/rpm build scripts are also changed
to do git submodule update.
Signed-off-by: Yonghong Song <yhs@fb.com>

The kernel libbpf library
  https://github.com/torvalds/linux/tree/master/tools/lib/bpf
provides common functionality for handling bpf programs, maps,
ELF parsing, probing, BTF manipulation, etc. The library has
been recommended to the community as a common library for
bpf/btf handling. A separate repository
  https://github.com/libbpf/libbpf
has been created for this purpose.

This patch brings in libbpf to bcc. The following command
is executed to crate a libbpf submodule at src/cc/libbpf:
  git submodule add https://github.com/libbpf/libbpf.git src/cc/libbpf
Signed-off-by: Yonghong Song <yhs@fb.com>

This fixes the format string used to parse the major and minor
device fields in /proc/self/maps. These fields have hexadecimal
values and hence cannot be parsed as unsigned integers.

Fixes: 51480d05 ("implement free_bcc_memory() API (#2097)")
Reported-by: Nageswara R Sastry rnsastry@linux.vnet.ibm.com
Signed-off-by: Sandipan Das <sandipan@linux.ibm.com>

Across the repo (see libbpf.h, api/BPF.h) we always use the custom header search include, to make it easier to work with different build environments.

Some users uses `BPFModule` directly instead of C++ / Python API, and
they would like to have similar interface of free BCC `.text` memory

Unify shebangs under `#!/usr/bin/python`