1. 10 Jan, 2019 2 commits
    • yonghong-song's avatar
      remove unused extern variables (#2125) · ae839790
      yonghong-song authored
      On fc29, compiling bcc from source, the following
      compiler warnings are seen:
        /home/yhs/work/bcc/src/cc/bcc_elf.c: In function ‘bcc_free_memory_with_file’:
        /home/yhs/work/bcc/src/cc/bcc_elf.c:802:36: warning: unused variable ‘_fini’ [-Wunused-variable]
             extern unsigned long _start, _fini;
                                          ^~~~~
      /home/yhs/work/bcc/src/cc/bcc_elf.c:802:28: warning: unused variable ‘_start’ [-Wunused-variable]
             extern unsigned long _start, _fini;
                                  ^~~~~~
      
      These unused externs are accidentally introduced by
      Commit 51480d05 ("implement free_bcc_memory() API").
      This patch removed them.
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      ae839790
    • Alexey Ivanov's avatar
      cmake/python: fix build race condition (#2111) · 9b3b1274
      Alexey Ivanov authored
      * python: remove MANIFEST
      
      * cmake/python: fix build race condition
      9b3b1274
  2. 09 Jan, 2019 4 commits
    • Jerome Marchand's avatar
      Translate arch into source directory when ARCH is set (#2122) · 28949f17
      Jerome Marchand authored
      When ARCH is not defined, bcc get the architecture from uname. It then
      modifies it to get the name of arch directory in linux source.
      
      When ARCH is defined however, it just copy it as is, without the
      translation to the arch directory. If for instance ARCH is set to
      x86_64, it tries to look into the include directory
      build/arch/x86_64/, which doesn't exist.
      
      It fixes the following issue:
      $ echo $ARCH
      x86_64
      $ /usr/share/bcc/tools/bashreadline
      In file included from <built-in>:2:
      In file included from /virtual/include/bcc/bpf.h:12:
      In file included from /lib/modules/4.18.0-49.el8.x86_64/build/include/linux/types.h:6:
      /lib/modules/4.18.0-49.el8.x86_64/build/include/uapi/linux/types.h:5:10: fatal error: 'asm/types.h' file
            not found
               ^~~~~~~~~~~~~
      1 error generated.
      Traceback (most recent call last):
        File "/usr/share/bcc/tools/bashreadline", line 51, in <module>
          b = BPF(text=bpf_text)
        File "/usr/lib/python3.6/site-packages/bcc/__init__.py", line 318, in __init__
          raise Exception("Failed to compile BPF text")
      Exception: Failed to compile BPF text
      28949f17
    • Takuma Kume's avatar
      tcpconnect: support uid option (#2118) · b181a8e7
      Takuma Kume authored
      support uid option in tcpconnect.
      b181a8e7
    • Xiaozhou Liu's avatar
      docs: fix parameters of bpf_trace_printk() (#2121) · e96836d9
      Xiaozhou Liu authored
      bpf_trace_printk() does not have `fmt_size` as the second parameter in BCC.
      e96836d9
    • detailyang's avatar
      tools: allow tcpaccept filter via PID (#2117) · 54044d51
      detailyang authored
      add missing FILTER in the bpf program.
      54044d51
  3. 07 Jan, 2019 2 commits
  4. 06 Jan, 2019 1 commit
  5. 04 Jan, 2019 2 commits
  6. 03 Jan, 2019 5 commits
  7. 01 Jan, 2019 1 commit
    • yonghong-song's avatar
      better error meessage for error "unknown opcode" (#2101) · dccc4f28
      yonghong-song authored
      fix issue #226
      
      The unknown opcode typically happens if the bpf
      program has an external reference which does not
      get resolved. Note bcc does not even preform
      relocations for maps as map_id is directly
      used in bpf problem through bpf_pseudo_fd()
      intrinsic.
      
      Instead of the error:
        bpf: Failed to load program: Invalid argument
        unknown opcode 00
      
      A little explanation is added like the below:
        HINT: The 'unknown opcode' can happen if you referencea global
        or static variable, or data in read only section.
        For example,'char *p = "hello"' will result in p referencing a
        read only section,and 'char p[] = "hello"' will have "hello"
        stored on the stack.
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      dccc4f28
  8. 31 Dec, 2018 1 commit
  9. 30 Dec, 2018 2 commits
    • Tim Douglas's avatar
      opensnoop: print flags, enable filtering (#2096) · d3583a8d
      Tim Douglas authored
      * opensnoop: print flags, enable filtering
      
      * Add docs, extended_fields option; filter flags in-kernel
      
      * Homogenize documentation
      
      * Add FLAGS to the FIELDS man page section
      d3583a8d
    • yonghong-song's avatar
      implement free_bcc_memory() API (#2097) · 51480d05
      yonghong-song authored
      The main purpose of this API is to proactively release llvm/clang
      .text memory which is brought in during compilation.
      bcc .text memory for some other functions, e.g., attach_tracepoint,
      bpf_prog_load, etc. can also be freed after all these tasks are done.
      
      Note that such memory is reclaimable in kernel since it has
      file backup. But certain applicaiton may want to reduce this
      memory immediately to satisfy constraints imposed by sysadmin, etc.
      
      The implementation uses madvise with MADV_DONTNEED.
      For the case where bcc is static linked into the binary,
      we do not really know the start and the end of memory regions
      used by bcc, so the implementation here bluntly returned
      all .text memory back to kernel. This will incur some performance
      overhead as later on executed instructions will need to bring
      back to memory again.
      
      For static linked library, instrumented RandomRead example,
      without this patch, the RSS memory before load is:
        VmRSS:     63644 kB
        RssAnon:           23876 kB
        RssFile:           39768 kB
        RssShmem:              0 kB
      
      After this patch,
        VmRSS:     34264 kB
        RssAnon:           23880 kB
        RssFile:           10384 kB
        RssShmem:              0 kB
      
      For shared library, a python unit test, test_free_llvm_memory.py, is
      added, which shows for a do-nothing bpf program, we have
        Before freeing llvm memory: RssFile:  43000 kB
        After  freeing llvm memory: RssFile:  11992 kB
      
      The RssFile reduction on Facebook internal applications
      also ranges in 30-40MB.
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      51480d05
  10. 27 Dec, 2018 1 commit
    • Prashant Bhole's avatar
      trace.py: fix compiler warning (#2094) · 05765eee
      Prashant Bhole authored
      Compiler shows warning "incompatible integer to pointer conversion
      initializing" while compiling bpf program.
      This patch adds necessary typecast when assigning PT_REGS_PARAM vaules
      to struct pt_regs pointer
      05765eee
  11. 26 Dec, 2018 1 commit
  12. 23 Dec, 2018 1 commit
    • torgil's avatar
      Make dependency on LLVM native target optional (#2080) · 61c063ae
      torgil authored
      * Make dependency on LLVM native target optional
      
      Adds an option ENABLE_LLVM_NATIVECODEGEN with default value ON.
      If set to off the "nativecodegen" llvm will not be enabled, thus
      reducing dependencies on needed libraries (reduced text size when
      building with statically linked libraries).
      
      Code that uses native target will not be compiled reducing text size.
      Currently this affects the rw_engine which needs the native target.
      
      BPF api "rw_engine_enabled" will have default value "true" if
      ENABLE_LLVM_NATIVECODEGEN="ON" and "false" if
      ENABLE_LLVM_NATIVECODEGEN="OFF"
      
      Not needed for BCC to work. It somehow brought in the interpreter and
      executionengine which is needed. Those features are added instead.
      
      * Remove garbage in code making it compile again
      
      * Remove interpreter and executionengine LLVM dependencies
      
      These doesn't seem to be needed on a Ubuntu 18.04 system (although
      executionengine is heavily used).
      
      Interpreter was added due to runtime dependency on ARM64. It brings in
      a dependency on ffi library.
      
      (.text._ZL10ffiTypeForPN4llvm4TypeE+0x3a): undefined reference to `ffi_type_float'
      (.text._ZL10ffiTypeForPN4llvm4TypeE+0x43): undefined reference to `ffi_type_void'
      (.text._ZL10ffiTypeForPN4llvm4TypeE+0x53): undefined reference to `ffi_type_pointer'
      (.text._ZL10ffiTypeForPN4llvm4TypeE+0x63): undefined reference to `ffi_type_double'
      (.text._ZL10ffiTypeForPN4llvm4TypeE+0x78): undefined reference to `ffi_type_sint8'
      (.text._ZL10ffiTypeForPN4llvm4TypeE+0x83): undefined reference to `ffi_type_sint16'
      (.text._ZL10ffiTypeForPN4llvm4TypeE+0x93): undefined reference to `ffi_type_sint64'
      (.text._ZL10ffiTypeForPN4llvm4TypeE+0xb3): undefined reference to `ffi_type_sint32'
      /usr/lib/llvm-6.0/lib/libLLVMInterpreter.a
      61c063ae
  13. 21 Dec, 2018 1 commit
  14. 19 Dec, 2018 1 commit
  15. 18 Dec, 2018 1 commit
  16. 17 Dec, 2018 1 commit
    • Jerome Marchand's avatar
      dbslower: fix a python3 bytes/string issue int the -x option · bffd94f3
      Jerome Marchand authored
      In python3, the find method requires a bytes-like object. It fixes the
      following error:
      
      $ dbslower mysql -x $(which mysqld)
      Traceback (most recent call last):
        File "/usr/share/bcc/tools/dbslower", line 72, in <module>
          if mysql_func_name.find("COM_DATA") >= 0:
      TypeError: a bytes-like object is required, not 'str'
      
      Also the -x option is currently undocumented in the man page and the
      example file. So let's ix that too.
      bffd94f3
  17. 14 Dec, 2018 1 commit
  18. 13 Dec, 2018 1 commit
  19. 12 Dec, 2018 3 commits
  20. 10 Dec, 2018 2 commits
  21. 06 Dec, 2018 3 commits
    • William Cohen's avatar
      Wcohen/efficiency (#2063) · 218f7482
      William Cohen authored
      * Reduce instrumentation overhead with the sys_enter and sys_exit tracepoints
      
      The ucalls script initially used kprobes and kretprobes on each of the
      hundreds of syscalls functions in the system.  This approach causes a
      large number of probes to be set up at the start and removed at the
      conclusion of the script's execution resulting in slow start up.
      
      Like the syscount.py script the ucall syscall instrumentation has been
      modified to use the sys_enter and sys_exit tracepoints.  This only
      requires the installation and removal of one or two tracepoints to
      implement and results in much shorter times to start and stop the
      ucalls script.
      
      Another benefit of this change is syscalls on newer kernels will be
      monitored with the "-S" option.  The regular expression used to find
      the locations for the kprobes and kretprobes for all the possible
      syscall functions would not would match the syscall function naming
      convention in newer kernels.
      
      * Update ucalls_examples.txt to match current "-S" option output
      
      * Add required "import subprocess" and remove unneeded "global syscalls"
      
      * Factor out the syscall_name code into a separate python module syscall.py
      
      Multiple scripts are going to find the syscall_name() function useful
      when using the syscall tracepoints.  Factoring out this code into a
      separate python module avoids having to replicate this code in
      multiple scripts.
      
      * Use the syscall_name() function in syscount.py to make it more compact.
      
      * Update the default syscall mappings and the way that they were generated
      
      The default table was missing some newer syscall mapping. Regenerated
      the table using the syscallent.h file from Fedora 30
      strace-4.25-1.fc30.src.rpm.  Also updated the comment with the command
      actually used to generate the mappings.
      
      * Add license information and upsdate the syscalls
      
      The default x86_64 syscall dictionary mapping syscalls numbers to
      names has been updated. The following syscall x86_64 names have been
      updated:
      
          18: b"pwrite64",
          60: b"exit",
          166: b"umount2",
      
      The following syscall x86_64 have been added:
      
          313: b"finit_module",
          314: b"sched_setattr",
          315: b"sched_getattr",
          316: b"renameat2",
          317: b"seccomp",
          318: b"getrandom",
          319: b"memfd_create",
          320: b"kexec_file_load",
          321: b"bpf",
          322: b"execveat",
          323: b"userfaultfd",
          324: b"membarrier",
          325: b"mlock2",
          326: b"copy_file_range",
          327: b"preadv2",
          328: b"pwritev2",
          329: b"pkey_mprotect",
          330: b"pkey_alloc",
          331: b"pkey_free",
          332: b"statx",
          333: b"io_pgetevents",
          334: b"rseq",
      
      * Eliminate stderr output and use of shell features
      
      Redirect all stderr output so it isn't seen.  Also avoid use of the
      shell pipe and tail command.  Just strip off the first line in the
      python code instead.
      
      * Update lib/ucalls.py smoke test to required linux-4.7
      
      The use of tracepoints in the ucalls.py requires linux-4.7. Changed
      the test to only run with a suitable kernel.  The libs/ucalls.py
      script is no longer inserting hundreds of kprobes and is much faster
      as a result, so removed the timeout adjustment and the comment about
      it being slow.
      218f7482
    • yonghong-song's avatar
      Revert "Remove duplicate code from xdp_drop_count.py (#2049)" (#2069) · f3fd8e30
      yonghong-song authored
      This reverts commit 5b76047f.
      
      The code is actually not duplicated. It is used to process
      double vlan's. See comments in:
        https://github.com/iovisor/bcc/pull/1493
      f3fd8e30
    • yonghong-song's avatar
      fix compilation error with latest llvm/clang (#2068) · 1f6ef8ce
      yonghong-song authored
      With latest llvm/clang, we have the following compilation errors:
      
          /home/yhs/work/bcc2/src/cc/frontends/clang/b_frontend_action.cc:
            In member function ‘bool ebpf::BTypeVisitor::VisitVarDecl(clang::VarDecl*)’:
          /home/yhs/work/bcc2/src/cc/frontends/clang/b_frontend_action.cc:1130:52:
            error: no matching function for call to ‘clang::Expr::EvaluateAsInt(llvm::APSInt&, clang::ASTContext&)’
                     if (I->getInit(idx)->EvaluateAsInt(res, C)) {
                                                          ^
          /home/yhs/work/bcc2/src/cc/frontends/clang/b_frontend_action.cc:1130:52: note: candidate is:
          In file included from /home/yhs/work/llvm/build/install/include/clang/AST/Attr.h:19:0,
                       from /home/yhs/work/llvm/build/install/include/clang/AST/TypeLoc.h:18,
                       from /home/yhs/work/llvm/build/install/include/clang/AST/ASTTypeTraits.h:24,
                       from /home/yhs/work/llvm/build/install/include/clang/AST/ASTContext.h:18,
                       from /home/yhs/work/bcc2/src/cc/frontends/clang/b_frontend_action.cc:23:
          /home/yhs/work/llvm/build/install/include/clang/AST/Expr.h:604:8:
            note: bool clang::Expr::EvaluateAsInt(clang::Expr::EvalResult&, const clang::ASTContext&,
                    clang::Expr::SideEffectsKind) const
             bool EvaluateAsInt(EvalResult &Result, const ASTContext &Ctx,
              ^
      
      The error is introduced by the following commit:
      https://reviews.llvm.org/rL348053
      
      Basically, the clang FieldDecl method EvaluateAsInt signature got
      changed and there is no compatible way in the llvm/clang
      to also work in the old versions. So this patch just provided
      a new implementation to get field value for llvm version 8 and later.
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      1f6ef8ce
  22. 05 Dec, 2018 1 commit
  23. 28 Nov, 2018 1 commit
    • William Cohen's avatar
      Update the tutorial to match what is currently in the examples (#2061) · a07ab90f
      William Cohen authored
      There have been some updates the code in the examples.  The tutorial
      text should match up with the current example code. In particular we
      want to make ensure that the probe for the syscall functions match up
      with what is currently needed for newer kernels that do not have
      syscall functions uniformly start with "sys_".
      a07ab90f
  24. 27 Nov, 2018 1 commit
    • Gerald Combs's avatar
      tcpstates: Add systemd journal logging. (#2058) · abdca97b
      Gerald Combs authored
      * tcpstates: Add systemd journal logging.
      
      Add a -Y/--journal flag to tcpstates.py, which logs events to the
      systemd journal.
      
      * tcpstates: Document systemd journal logging.
      
      Update tcpstates_example.txt and tcpstates.8 to include the "-Y" flag.
      abdca97b