1. 21 Nov, 2018 1 commit
  2. 20 Nov, 2018 2 commits
  3. 18 Nov, 2018 1 commit
    • Lecopzer's avatar
      Fix some compiler warning (#2047) · 62bc2259
      Lecopzer authored
      * Allow unused return value in cc source
      
      With llvm-7.0.0,
      some annoying warning messeges are raised:
      
      	/home/lecopzer/workspace/bcc/src/cc/libbpf.c:456:3: warning: ignoring return value of ‘fgets’, declared with attribute warn_unused_result [-Wunused-result]
         	fgets(fmt, sizeof(fmt), f); // pos
         	^~~~~~~~~~~~~~~~~~~~~~~~~~
      
      /home/lecopzer/workspace/bcc/src/cc/libbpf.c: In function ‘bpf_prog_get_tag’:
      /home/lecopzer/workspace/bcc/src/cc/libbpf.c:456:3: warning: ignoring return value of ‘fgets’, declared with attribute warn_unused_result [-Wunused-result]
         fgets(fmt, sizeof(fmt), f); // pos
         ^~~~~~~~~~~~~~~~~~~~~~~~~~
      ...
      
      /home/lecopzer/workspace/bcc/tests/cc/utils.cc: In function ‘int cmd_scanf(const char*, const char*, ...)’:
      /home/lecopzer/workspace/bcc/tests/cc/utils.cc:30:10: warning: ignoring return value of ‘int vfscanf(FILE*, const char*, __va_list_tag*)’, declared with attribute warn_unused_result [-Wunused-result]
         vfscanf(pipe, fmt, args);
         ~~~~~~~^~~~~~~~~~~~~~~~~
      
      Let get rid of them by adding -Wno-unused-result.
      
      * cc: Fix comparison between signed and unsigned value
      
      With llvm-7.0.0:
      
      /home/lecopzer/workspace/bcc/src/cc/common.cc: In function ‘std::__cxx11::string ebpf::get_pid_exe(pid_t)’:
      /home/lecopzer/workspace/bcc/src/cc/common.cc:60:11: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
         if (res >= sizeof(exe_path))
             ~~~~^~~~~~~~~~
      
      As the declaration of `exe_path` is `char exe_path[4096]`,
      the `sizeof(exe_path)` would always return 4096 (unsigned), so it's safe
      to static cast to `int` unless it's larger than 2^31 - 1.
      62bc2259
  4. 16 Nov, 2018 1 commit
    • olsajiri's avatar
      Add shmsnoop/sofdsnoop tools v2 (#2045) · b5114229
      olsajiri authored
      * tools: Add shmsnoop to spy on shm* syscalls
      
      Adding shmsnoop tool to trace System V shared memory
      syscalls: shmget, shmat, shmdt, shmctl
      
        # ./shmsnoop.py
        PID    COMM                SYS              RET ARGs
        19813  server           SHMGET            10000 key: 0x78020001, size: 20, shmflg: 0x3b6 (IPC_CREAT|0666)
        19813  server            SHMAT     7f1cf8b1f000 shmid: 0x10000, shmaddr: 0x0, shmflg: 0x0
        19816  client           SHMGET            10000 key: 0x78020001, size: 20, shmflg: 0x1b6 (0666)
        19816  client            SHMAT     7f4fd8ee7000 shmid: 0x10000, shmaddr: 0x0, shmflg: 0x0
        19816  client            SHMDT                0 shmaddr: 0x7f4fd8ee7000
        19813  server            SHMDT                0 shmaddr: 0x7f1cf8b1f000
        19813  server           SHMCTL                0 shmid: 0x10000, cmd: 0, buf: 0x0
      
      Every call the shm* syscall (SHM column) is displayed
      on separate line together with process info (PID/COMM
      columns) and argument details: return value (RET column)
      and syscall arguments (ARGs column).
      
      The ARGs column contains 'arg: value' couples that represent
      given syscall arguments as described in their manpage.
      
      It supports standard options to filter on pid/tid,
      to specify duration of the trace and command name
      filter, like:
      
          ./shmsnoop           # trace all shm*() syscalls
          ./shmsnoop -T        # include timestamps
          ./shmsnoop -p 181    # only trace PID 181
          ./shmsnoop -t 123    # only trace TID 123
          ./shmsnoop -d 10     # trace for 10 seconds only
          ./shmsnoop -n main   # only print process names containing "main"
      
      * tools: Add sofdsnoop to spy on fds passed through socket
      
      The sofdsnoop traces FDs passed through unix sockets.
      
        # ./sofdsnoop
        ACTION TID    COMM             SOCKET                    FD    NAME
        SEND   2576   Web Content      24:socket:[39763]         51    /dev/shm/org.mozilla.ipc.2576.23874
        RECV   2576   Web Content      49:socket:[809997]        51
        SEND   2576   Web Content      24:socket:[39763]         58    N/A
        RECV   2464   Gecko_IOThread   75:socket:[39753]         55
      
      Every file descriptor that is passed via unix sockets os displayed
      on separate line together with process info (TID/COMM columns),
      ACTION details (SEND/RECV), file descriptor number (FD) and its
      translation to file if available (NAME).
      
      examples:
          ./sofdsnoop           # trace file descriptors passes
          ./sofdsnoop -T        # include timestamps
          ./sofdsnoop -p 181    # only trace PID 181
          ./sofdsnoop -t 123    # only trace TID 123
          ./sofdsnoop -d 10     # trace for 10 seconds only
          ./sofdsnoop -n main   # only print process names containing "main"
      b5114229
  5. 15 Nov, 2018 2 commits
  6. 13 Nov, 2018 1 commit
    • Sandipan Das's avatar
      Fix funcslower stack traces when using arguments (#2040) · 6bbdb9c6
      Sandipan Das authored
      This reorders the struct data_t members so that the definition
      is consistent across the c code and the corresponding python
      ct.Structure.
      
      Upon running the script with the arguments option, the user and
      kernel stack ids read from the stack maps are incorrect as they
      are read off the wrong structure offsets. When a stack walk is
      attempted on an incorrect stack id, we end up with a KeyError.
      
      This has been verified as shown below.
      
        $ sudo ./funcslower.py c:inet_pton -u1 -a1 -UK
        $ ping -6 ::1
      
      Before:
        Tracing function calls slower than 1 us... Ctrl+C to quit.
        COMM           PID    LAT(us)             RVAL FUNC ARGS
        ping           33541    47.93                1 c:inet_pton 0xfffffff2000001a0
        Traceback (most recent call last):
          File "_ctypes/callbacks.c", line 315, in 'calling callback function'
          File "/usr/lib/python2.7/site-packages/bcc/table.py", line 573, in raw_cb_
            callback(cpu, data, size)
          File "./funcslower.py", line 337, in print_event
            print_stack(event)
          File "./funcslower.py", line 301, in print_stack
            user_stack = stack_traces.walk(event.user_stack_id)
          File "/usr/lib/python2.7/site-packages/bcc/table.py", line 768, in walk
            return StackTrace.StackWalker(self[self.Key(stack_id)], resolve)
          File "/usr/lib/python2.7/site-packages/bcc/table.py", line 212, in __getitem__
            raise KeyError
        KeyError
      
      After:
        Tracing function calls slower than 1 us... Ctrl+C to quit.
        COMM           PID    LAT(us)             RVAL FUNC ARGS
        ping           34672    48.20                1 c:inet_pton 0xa
            gaih_inet.constprop.7
            [unknown]
            getaddrinfo
            [unknown]
            generic_start_main.isra.0
            __libc_start_main
      
      Fixes: 925bac87 ("Adding user and kernel stack frames option to funcslower")
      Signed-off-by: default avatarSandipan Das <sandipan@linux.ibm.com>
      6bbdb9c6
  7. 10 Nov, 2018 2 commits
    • yonghong-song's avatar
      fix verifier errors in http_filter example (#2039) · 60b0166f
      yonghong-song authored
      Fix issue #2035.
      
      For code like
          int i;
          int j = 0;
          const int last_index = payload_offset + 7;
          for (i = payload_offset ; i < last_index ; i++) {
                     p[j] = load_byte(skb , i);
      
      Here, the payload_offset is unknown. llvm 7.0 and trunk
      compiler seems generating code like
         p[0] = load_byte(skb, payload_offset)
         if (payload_offset + 1 < last_index) {
            p[1] = ...
            ...
            p[6] = ...
         } else {
            /* do nothing */
         }
         /* accessing p[0], p[1], ..., p[6] */
      
      The compiler did the above transformation because the potential
      overflow for last_index and/or payload_offset + 1 in which case
      compiler preserved both branches.
      
      This caused a problem for verifier as in the else branch
      p[1] is not assigned and the verifier will reject the program.
      
      Changing the loop to simply iterate from 0 to 6 fixed the problem.
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      60b0166f
    • Oliver Mannion's avatar
      install docs - on ubuntu bionic use llvm 6.0 because it doesn't have llvm 3.7 packages (#2038) · 726c60f6
      Oliver Mannion authored
      install docs - on ubuntu bionic use llvm 6.0
      726c60f6
  8. 09 Nov, 2018 3 commits
    • Sandipan Das's avatar
      Fix uprobes on powerpc64 (#2032) · bbd4180c
      Sandipan Das authored
      * Use correct entry point for uprobes on powerpc64
      
      For powerpc64 (big endian), the address of a function is the
      address of the corresponding function descriptor. While the
      actual functions reside in the ".text" section, the function
      descriptors are present in the ".opd" section. According to
      the ABI, each descriptor is a tri-doubleword data structure
      where the first doubleword is the actual entry point address.
      
      The symbol table entries do not list actual entry points but
      instead provide the location of the function descriptor. So,
      when attaching a probe, the location should be changed to the
      actual entry point by peeking into the function descriptor.
      
      This has been verified as shown below.
      
        $ readelf -S /usr/lib64/power8/libc-2.26.so | grep -A1 ".opd"
          [30] .opd              PROGBITS         0000000000213648  00203648
               000000000000bcb8  0000000000000000  WA       0     0     8
      
      The first column shows the index of the ".opd" section.
      
        $ readelf -s /usr/lib64/power8/libc-2.26.so | grep "inet_pton$"
          3405: 000000000021d168    96 FUNC    LOCAL  DEFAULT   30 __inet_pton
          3990: 000000000021d168    96 FUNC    LOCAL  DEFAULT   30 __GI___inet_pton
          5167: 000000000021d168    96 FUNC    LOCAL  DEFAULT   30 __GI_inet_pton
          6514: 000000000021d168    96 FUNC    WEAK   DEFAULT   30 inet_pton
      
      The seventh column shows the index of the section to which the
      symbols belong. This implies that all of these symbols are from
      the ".opd" section.
      
        $ objdump -d --section=.opd /usr/lib64/power8/libc-2.26.so | grep -A5 "inet_pton>:"
        000000000021d168 <inet_pton>:
          21d168:       00 00 00 00     .long 0x0
          21d16c:       00 17 2b 40     .long 0x172b40
          21d170:       00 00 00 00     .long 0x0
          21d174:       00 22 73 00     .long 0x227300
      
        $ objdump -d /usr/lib64/power8/libc-2.26.so | grep -A5 "inet_pton>:"
        0000000000172b40 <.__inet_pton>:
          172b40:       7c 08 02 a6     mflr    r0
          172b44:       fb c1 ff f0     std     r30,-16(r1)
          172b48:       fb e1 ff f8     std     r31,-8(r1)
          172b4c:       7c 7f 1b 78     mr      r31,r3
          172b50:       7c 83 23 78     mr      r3,r4
      
      The first doubleword in the descriptor of "inet_pton" gives the
      actual entry point address i.e. 0x172b40. So, the probe must be
      attached here and not 0x21d168.
      
        $ sudo trace "c:inet_pton" -U
        PID     TID     COMM            FUNC
        40769   40769   ping            inet_pton
                __GI___inet_pton+0x0 [libc-2.26.so]
                gaih_inet.constprop.7+0xf4c [libc-2.26.so]
                __GI_getaddrinfo+0x15c [libc-2.26.so]
                [unknown] [ping]
                generic_start_main.isra.0+0x150 [libc-2.26.so]
                __libc_start_main+0xbc [libc-2.26.so]
      
        $ ping -6 ::1
        PING ::1(::1) 56 data bytes
        64 bytes from ::1: icmp_seq=1 ttl=64 time=0.271 ms
        64 bytes from ::1: icmp_seq=2 ttl=64 time=0.039 ms
        ^C
        --- ::1 ping statistics ---
        2 packets transmitted, 2 received, 0% packet loss, time 1058ms
        rtt min/avg/max/mdev = 0.039/0.155/0.271/0.116 ms
      
      Previously, the event was not triggered upon running ping.
      Signed-off-by: default avatarSandipan Das <sandipan@linux.ibm.com>
      
      * Use correct entry point for uprobes on powerpc64le
      
      For powerpc64le, functions have a Global Entry Point (GEP)
      and a Local Entry Point (LEP). When using the GEP, there
      are some additional instructions at the beginning of the
      function that setup the TOC pointer. However, for all local
      calls, the TOC pointer is not required and a function can
      be called into directly via the LEP.
      
      While placing a uprobe, we should always prefer the LEP as
      the probe location since this will be encountered for any
      call through either the GEP or the LEP. Currently, the GEP
      is used as the probe location and hence the corresponding
      event is never triggered when the function is called via
      it's LEP.
      
      Information about the LEP can be obtained from the st_other
      field of an Elf symbol. While this field typically provides
      visibility information, the three most significant bits can
      provide additional information about the offset of the Local
      Entry Point (LEP) from the Global Entry Point (GEP) for any
      symbol in case of powerpc64le.
      
      This has been verified as shown below.
      
        $ readelf -s /usr/lib64/libc-2.27.so | grep "inet_pton "
        3522: 0000000000164610   104 FUNC    LOCAL  DEFAULT   11 __inet_pton  [<localentry>: 8]
        4188: 0000000000164610   104 FUNC    LOCAL  DEFAULT   11 __GI___inet_pton     [<localentry>: 8]
        5528: 0000000000164610   104 FUNC    LOCAL  DEFAULT   11 __GI_inet_pton       [<localentry>: 8]
        6925: 0000000000164610   104 FUNC    WEAK   DEFAULT   11 inet_pton    [<localentry>: 8]
      
        $ sudo trace "c:inet_pton" -U
        PID     TID     COMM            FUNC
        25383   25383   ping            inet_pton
                __GI___inet_pton+0x8 [libc-2.27.so]
                gaih_inet.constprop.7+0x1040 [libc-2.27.so]
                getaddrinfo+0x164 [libc-2.27.so]
                [unknown] [ping]
                generic_start_main.isra.0+0x138 [libc-2.27.so]
                __libc_start_main+0xc4 [libc-2.27.so]
      
        $ ping -6 ::1
        PING ::1(::1) 56 data bytes
        64 bytes from ::1: icmp_seq=1 ttl=64 time=0.140 ms
        64 bytes from ::1: icmp_seq=2 ttl=64 time=0.029 ms
        ^C
        --- ::1 ping statistics ---
        2 packets transmitted, 2 received, 0% packet loss, time 1022ms
        rtt min/avg/max/mdev = 0.029/0.084/0.140/0.056 ms
      
      Previously, the event was not triggered upon running ping.
      Signed-off-by: default avatarSandipan Das <sandipan@linux.ibm.com>
      bbd4180c
    • Teng Qin's avatar
      Merge pull request #2037 from sandip4n/lua-dev · 64a709da
      Teng Qin authored
      Fix the lua standalone test
      64a709da
    • Oriol Arcas's avatar
      Print name of failing program (#2036) · 4ce0b108
      Oriol Arcas authored
      When kprobe/kreprobe/tracepoint attachment fails, print the name of the
      failing function and its target.
      Signed-off-by: default avatarOriol Arcas <oriol@starflownetworks.com>
      4ce0b108
  9. 08 Nov, 2018 2 commits
  10. 05 Nov, 2018 2 commits
  11. 03 Nov, 2018 1 commit
  12. 02 Nov, 2018 1 commit
  13. 01 Nov, 2018 2 commits
    • Jerome Marchand's avatar
      covscan: fix miscellaneaous errors (#2003) · 415bd4e4
      Jerome Marchand authored
      * Coverity #def53: COPY_PASTE_ERROR
      
      * Coverity #def18: DC.STREAM_BUFFER. Double-check max length of dev
      
      * Coverity #def44: MISSING_BREAK. This looks like it should be here
      
      * Coverity #def67: STRING_NULL: potential OOB read if 0 bytes read.
      
      * Coverity #def66: FORWARD_NULL: potential null ptr deref
      
      * Coverity #def17: RESOURCE_LEAK: missing free()
      
      * Dont free the result of dirname
      
      dirname() may return pointers to statically allocated memory. Don't
      free the pointer it returns.
      415bd4e4
    • Ildar Musin's avatar
      Fix USDT semaphore address calculation for position independent executables... · b998421b
      Ildar Musin authored
      Fix USDT semaphore address calculation for position independent executables (resolves #1998) (#2023)
      
      Fix USDT semaphore address calculation for position independent executables
      b998421b
  14. 30 Oct, 2018 3 commits
    • yonghong-song's avatar
      allow packed structure in bpf program in python API (#2020) · 9410c86d
      yonghong-song authored
      Fix issue #2017.
      
      For python programs, the map data passed from C++ library
      is parsed through the key/value desc types which are
      generated by C++ json map declaration visitor.
      
      The map declaration visitor visits the map key/value
      declaration types and generate a string to represent
      the type, which is later used by python to reconstruct
      the cttype.
      
      The map declaration already tries to add all the padding
      to the type string in order to make sure C++ and python
      see the same layout.
      
      This patch further added packed support such that if
      C++ json map visitor has applied padding, the python
      type reconstructor is free to add _pack_=1 for structure
      type since the structure is already packed.
      
      For example, for a type,
        struct t { char a; int b; }
      the structure after json visitor will look like
        struct t { char a; char __pad[3]; int b; }
      
      If the type is
        struct t { char a; int b; } __packed;
      the structure after json visitor will look like
        struct t { char a; int b; }
      
      In either case, it will be okay to add __packed attribute
      for the type generated by json map visitor in order to
      match the original declaration.
      
      Thanks Chaitanya for filing the issue and providing the test case!
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      9410c86d
    • yonghong-song's avatar
      Merge pull request #2021 from cyphar/32bit-uint128 · b81dcb7c
      yonghong-song authored
      frontends: disable uint128_t on 32-bit builds
      b81dcb7c
    • Aleksa Sarai's avatar
      frontends: disable uint128_t on 32-bit builds · 329faafd
      Aleksa Sarai authored
      On 32-bit, __uint128_t is generally not available. Removing it fixes
      builds on those systems.
      Signed-off-by: default avatarAleksa Sarai <cyphar@cyphar.com>
      329faafd
  15. 29 Oct, 2018 2 commits
    • yonghong-song's avatar
      Merge pull request #2019 from iovisor/yhs_dev · 895aefda
      yonghong-song authored
      correct certain tracepoint types in tp frontend action
      895aefda
    • Yonghong Song's avatar
      use proper sync for function pointers · 1589a4f0
      Yonghong Song authored
      This is a correction for previous patch where
      function pointer is not proper specified
        static int bpf_map_push_elem(struct bpf_map *map, const void *value, u64 flags) =
           (void *) BPF_FUNC_map_push_elem;
      and some old gcc compiler (e.g., ubuntu 16.04) will complain
      with error:
        Illegal initializer (only variables can be initialized)
      The correct way is
        static int (*bpf_map_push_elem)(struct bpf_map *map, const void *value, u64 flags) =
           (void *) BPF_FUNC_map_push_elem;
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      1589a4f0
  16. 28 Oct, 2018 3 commits
    • Yonghong Song's avatar
      correct certain tracepoint types in tp frontend action · 0e02fec8
      Yonghong Song authored
      Fix issue #2010.
      
      Alastair reported some missing cases in tp frontend action
      where certain types are not adjusted.
      For example, for tracepoint syscalls:sys_enter_kill,
      the kernel format:
              ...
              field:int __syscall_nr; offset:8;       size:4; signed:1;
              field:pid_t pid;        offset:16;      size:8; signed:0;
              field:int sig;  offset:24;      size:8; signed:0;
      
      The size for "pid_t pid" is 8 bytes, but the kernel pid_t
      is "int", so it is needed to change its type to "s64"
      to be consistent to its size.
      
      This patch also added conversion for gid_t and uid_t as Alastair
      discovered that they are also used in some tracepoints with size 8.
      For type gid_t and uid_t, the corresponding kernel type is "unsigned int",
      so it should be converted to "u64".
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      0e02fec8
    • 4ast's avatar
      Merge pull request #2018 from iovisor/yhs_dev · f3fc87aa
      4ast authored
      sync with latest net-next
      f3fc87aa
    • Yonghong Song's avatar
      sync with latest net-next · 2a68bd8b
      Yonghong Song authored
      Sync bpf uapi header bpf.h with latest net-next
      and update related documentation.
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      2a68bd8b
  17. 15 Oct, 2018 2 commits
    • 4ast's avatar
      Merge pull request #2011 from iovisor/yhs_dev · afd00154
      4ast authored
      fix bcc build failure with latest llvm/clang trunk
      afd00154
    • Yonghong Song's avatar
      fix bcc build failure with latest llvm/clang trunk · 331ab236
      Yonghong Song authored
      The clang commit
        commit 247be3e41c41201641af01888ff0d84484c4e0f4
        Author: Simon Pilgrim <llvm-dev@redking.me.uk>
        Date:   Fri Oct 12 19:14:43 2018 +0000
      
          Fix a clang::driver::ArgStringList usage I missed in rL344398. NFCI.
      removed
        using ArgStringList = llvm::opt::ArgStringList;
      in clang:include/clang/Driver/Job.h,
      and makes bcc compilation failure.
      
      Using llvm::opt::ArgStringList fixed the issue and should
      still work with previous llvm versions.
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      331ab236
  18. 12 Oct, 2018 1 commit
    • jeromemarchand's avatar
      Tools fixes (#2009) · 09f9d3c8
      jeromemarchand authored
      * tools: the argument of get_kprobe_functions() should be a bytes object
      
      It fixes the following error:
      Traceback (most recent call last):
        File "/usr/share/bcc/tools/ext4dist", line 189, in <module>
          if BPF.get_kprobe_functions('ext4_file_read_iter'):
        File "/usr/lib/python3.6/site-packages/bcc/__init__.py", line 519, in get_kprobe_functions
          if (t.lower() in [b't', b'w']) and re.match(event_re, fn) \
        File "/usr/lib64/python3.6/re.py", line 172, in match
          return _compile(pattern, flags).match(string)
      TypeError: cannot use a string pattern on a bytes-like object
      
      * tools: fix freeze in offwaketime
      
      Since commit 47cecb65, the sigint signal is ignored
      unconditionally which prevents offwaketime to be stopped by
      pressing Ctrl-C when run without a specified duration.
      09f9d3c8
  19. 11 Oct, 2018 1 commit
  20. 10 Oct, 2018 2 commits
    • yonghong-song's avatar
      sync with latest net-next bpf headers (#2001) · d4b23876
      yonghong-song authored
      Sync compat bpf.h and virtual_bpf.h with latest net-next versions
      
      Also add a barrier for the test_brb bpf program like below
               meta.prog_id = skb->cb[0];
      +        asm volatile("" ::: "memory");
               meta.rx_port_id = skb->cb[1];
      so two cb read does not become a 8 byte load which is not
      allowed for ubuntu 16.04 kernel (4.4 based). The 8 byte load
      is allowed for skb->cb's for later kernels.
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      d4b23876
    • Marko Myllynen's avatar
      utools Tcl support (#2005) · 9f3662ee
      Marko Myllynen authored
      * Add perl* utools man page links
      
      Matches other languages, related to commit 9162be45.
      
      * uthreads: explicitly list C as supported
      
      uthreads supports C (pthreads) thread tracing, make this explicit.
      
      Earlier use of -l java and -l none unchanged.
      
      Add the corresponding man page and usage examples.
      
      * Add Tcl support for ucalls / uflow / uobjnew / ustat
      
      For some reason we need to use proc__args instead of proc__entry
      in uflow.py to get similar results as with e.g. Python.
      9f3662ee
  21. 09 Oct, 2018 3 commits
  22. 07 Oct, 2018 1 commit
  23. 05 Oct, 2018 1 commit