1. 15 Jun, 2016 2 commits
  2. 13 Jun, 2016 3 commits
    • Brenden Blanco's avatar
      Merge pull request #563 from qmonnet/debug-output · cfac8da4
      Brenden Blanco authored
      libbpf.c: Prepend BPF syscall error to log_bug when debug is enabled.
      cfac8da4
    • Quentin Monnet's avatar
      python: With DEBUG_BPF, print BPF error even if log_buf is not empty. · 4ddedda5
      Quentin Monnet authored
      When BPF syscall fails and DEBUG_FLAG has been provided to a Python
      caller script, the BPF error string would be printed only if log_buf had
      not been filled by the kernel. This commit removes this requirement,
      printing the error string even if log_buf is not empty.
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@6wind.com>
      4ddedda5
    • Quentin Monnet's avatar
      python: Print BPF syscall error if DEBUG_BPF is on but log_buf is empty. · bb699887
      Quentin Monnet authored
      Commit 759029fe provided an option to
      store the output from BPF syscall into a buffer (and not to print it
      systematically to standard output) on program load in libbpf.c.
      
      But doing so, it only stores the content of attr.log_buf, while the
      error string--resulting from a failed BPF syscall--is no more displayed
      when the DEBUG_BPF flag is used in the Python script responsible for
      converting and injecting the code.
      
      This commit proposes a fix for this bug by printing the error message
      (associated to the return value from the syscall) from the Python
      caller, when all the following conditions are met:
      
      - the syscall fails,
      - the DEBUG_BPF flag has been provided, and
      - log_buf is empty (has not been filled by kernel).
      
      Note: when DEBUG_BPF is not provided, the error string is printed in the
      C wrapper in libbpf.c (bpf_prog_load) anyway.
      
      Fixes: 759029fe ("Add option for custom log string to bpf_prog_load")
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@6wind.com>
      bb699887
  3. 08 Jun, 2016 1 commit
  4. 07 Jun, 2016 4 commits
  5. 06 Jun, 2016 3 commits
  6. 03 Jun, 2016 2 commits
  7. 02 Jun, 2016 2 commits
  8. 01 Jun, 2016 2 commits
  9. 29 May, 2016 1 commit
    • Jincheng Miao's avatar
      set use_errno to True for libbcc.so · f4a3e41a
      Jincheng Miao authored
      Currently, errno could not be accessed by libbcc.py, like:
      ```
      binding socket to 'eth0'
      cannot create raw socket: Operation not permitted
      Traceback (most recent call last):
        File "/home/ryan/Software/bcc/examples/networking/http_filter/http-parse-simple.py", line 73, in <module>
          BPF.attach_raw_socket(function_http_filter, interface)
        File "/usr/lib/python2.7/site-packages/bcc/__init__.py", line 330, in attach_raw_socket
          raise Exception("Failed to open raw device %s: %s" % (dev, errstr))
      Exception: Failed to open raw device eth0: Success
      ```
      The exception prints the errno as `Success`, instead of `EPERM`.
      
      This patch would fix this problem.
      Signed-off-by: default avatarJincheng Miao <jincheng.miao@gmail.com>
      f4a3e41a
  10. 27 May, 2016 1 commit
  11. 26 May, 2016 2 commits
  12. 24 May, 2016 6 commits
    • 4ast's avatar
      Merge pull request #551 from vmg/vmg/lua-para · f67a7af5
      4ast authored
      lua: Allow LuaJIT parameterized types in `open_perf_buffer`
      f67a7af5
    • Vicent Marti's avatar
    • 4ast's avatar
      Merge pull request #550 from markdrayton/trace-fields · 9bbe58f9
      4ast authored
      Use BPF_PERF_OUTPUT for fileslower and stacksnoop
      9bbe58f9
    • Mark Drayton's avatar
      266d6f69
    • Andrew Birchall's avatar
      7f0a6f80
    • Andrew Birchall's avatar
      Add user space stack traces to offcputime · 1f202e7b
      Andrew Birchall authored
      Summary:
      * generalize `KernelSymbolCache` into `SymbolCache` which supports user/kernel space symbols
      * create `BPF.sym()` for resolving user/kernel space symbols
      * `_ksym_cache` => `_sym_caches` and create `BPF._sym_cache()` to leverage the symbol caches
      * update `tools/offcputime.py` to print user space stack traces
      
      Test Plan:
      ```
      dev[bcc](abirchall_next): ~/bcc_run_tool.sh offcputime -U -k -f 1
      ERROR: Displaying user stacks for kernel threads doesn't make sense.
      devbig680[bcc](abirchall_next): ~/bcc_run_tool.sh offcputime -K -f 1 | grep python2 | head -n 1
      python2.7;system_call_fastpath;sys_futex;do_futex;futex_wait;futex_wait_queue_me;schedule 19
      dev[bcc](abirchall_next): ~/bcc_run_tool.sh offcputime -U -f 1 | grep python2 | head -n 1
      python2.7;clone;start_thread;t_bootstrap;PyEval_CallObjectWithKeywords;PyObject_Call;instancemethod_call;PyObject_Call;function_call;PyEval_EvalCodeEx;PyEval_EvalFrameEx;PyObject_Call;function_call;PyEval_EvalCodeEx;PyEval_EvalFrameEx;time_sleep;PyEval_RestoreThread 5
      devbig680[bcc](abirchall_next): ~/bcc_run_tool.sh offcputime -f 1 | grep python2 | head -n 1
      python2.7;system_call_fastpath;sys_futex;do_futex;futex_wait;futex_wait_queue_me;schedule;[unknown];__libc_start_main;Py_Main;RunModule;PyObject_Call;function_call;PyEval_EvalCodeEx;PyEval_EvalFrameEx;PyEval_EvalFrameEx;PyEval_EvalCodeEx;PyEval_EvalFrameEx;lock_PyThread_acquire_lock;PyEval_RestoreThread 39
      ```
      1f202e7b
  13. 23 May, 2016 1 commit
  14. 20 May, 2016 2 commits
  15. 19 May, 2016 1 commit
  16. 17 May, 2016 3 commits
    • Brendan Gregg's avatar
      Merge pull request #543 from abirchall/abirchall_release · c7f317de
      Brendan Gregg authored
      add --kernel-threads-only to tools/offcputime
      c7f317de
    • Andrew Birchall's avatar
      add --kernel-threads-only to tools/offcputime · 582b5dd2
      Andrew Birchall authored
      Summary:
      Adds `--kernel-threads-only` arg
      The kernel-threads-only arg is exclusive with pid/user-threads-only via `parser.add_mutually_exclusive_group`.
      The output message now indicates what we are tracing (pid/user threads/kernel threads/all threads).
      Removed the --verbose arg (unused).
      
      Test Plan:
      Run with combinations of the args; validate output looks sane:
      
      // test mutually exclusive group
      ```
      devbig680[bcc](abirchall_dev): ~/bcc_run_tool.sh offcputime -k -u 1
      [Running] /data/users/abirchall/bcc/tools/offcputime.py -k -u 1
      usage: offcputime.py [-h] [-p PID | -k | -u] [-v] [-f] [duration]
      offcputime.py: error: argument -u/--user-threads-only: not allowed with argument -k/--kernel-threads-only
      ```
      
      // kernel threads only
      ```
      devbig680[bcc](abirchall_dev): ~/bcc_run_tool.sh offcputime -f -k 1
      [Running] /data/users/abirchall/bcc/tools/offcputime.py -f -k 1
      swapper/21;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 11
      swapper/16;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 19
      swapper/22;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 20
      swapper/31;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 20
      swapper/23;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 67
      swapper/25;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 74
      ...
      ```
      `~/bcc_run_tool.sh offcputime -f --kernel-threads-only 1`
      
      // user threads only
      `~/bcc_run_tool.sh offcputime -f --user-threads-only 1`
      `~/bcc_run_tool.sh offcputime -f -u 1`
      
      // specific pid
      `~/bcc_run_tool.sh offcputime -f -p $(pidof hphpi) 1`
      ```
      devbig680[bcc](abirchall_dev): ~/bcc_run_tool.sh offcputime --pid $(pidof mcrouter) 10 | head
      [Running] /data/users/abirchall/bcc/tools/offcputime.py --pid 95929 10
      Tracing off-CPU time (us) of PID 95929 by kernel stack for 10 secs.
      ```
      
      Note that this last case (specific PID) doesn't appear to be working; I can debug that in a follow up commit.
      582b5dd2
    • 4ast's avatar
      Merge pull request #533 from abirchall/abirchall_dev · 09534d89
      4ast authored
      handle ENOMEM in tools/offcputime
      09534d89
  17. 16 May, 2016 4 commits
    • Andrew Birchall's avatar
      move ENOMEM comment; create --stack-storage-size arg; misc cleanup · 47d871f0
      Andrew Birchall authored
      Summary:
      * move ENOMEM comment from tools/offcputime.py to src/cc/export/helpers.h
      * create --stack-storage-size arg to allow the user to set the stack storage size
          * requires a positive_nonzero_int type checking function for argparse
      * clean up the rest of the args a bit
          * use parser.add_mutually_exclusive_group instead of manually handling the
            mutual exclusion
          * use positive_nonzero_int for duration arg
          * use positive_int for pid arg
      * only print a warning about increasing the storage size if at least one of the
        get_stackid errors was a -ENOMEM
      * remove the debug param (we can add this manually when testing)
      * fix a bug where all processes are traced when specifying pid of 0
      
      Test Plan:
      ```
      devbig680[bcc](no branch, rebasing abirchall_dev): ~/bcc_run_tool.sh offcputime --help
      [Running] /data/users/abirchall/bcc/tools/offcputime.py --help
      usage: offcputime.py [-h] [-u | -p PID] [-v] [-f]
                           [--stack-storage-size STACK_STORAGE_SIZE]
                           [duration]
      
      Summarize off-CPU time by kernel stack trace
      
      positional arguments:
        duration              duration of trace, in seconds
      
      optional arguments:
        -h, --help            show this help message and exit
        -u, --useronly        user threads only (no kernel threads)
        -p PID, --pid PID     trace this PID only
        -v, --verbose         show raw addresses
        -f, --folded          output folded format
        --stack-storage-size STACK_STORAGE_SIZE
                              the number of unique stack traces that can be stored
                              and displayed
      
      examples:
          ./offcputime             # trace off-CPU stack time until Ctrl-C
          ./offcputime 5           # trace for 5 seconds only
          ./offcputime -f 5        # 5 seconds, and output in folded format
          ./offcputime -u          # don't include kernel threads (user only)
          ./offcputime -p 185      # trace fo PID 185 only
      devbig680[bcc](no branch, rebasing abirchall_dev): ~/bcc_run_tool.sh offcputime -f 0
      [Running] /data/users/abirchall/bcc/tools/offcputime.py -f 0
      usage: offcputime.py [-h] [-u | -p PID] [-v] [-f]
                           [--stack-storage-size STACK_STORAGE_SIZE]
                           [duration]
      offcputime.py: error: argument duration: must be positive and nonzero
      devbig680[bcc](no branch, rebasing abirchall_dev): ~/bcc_run_tool.sh offcputime -f -1
      [Running] /data/users/abirchall/bcc/tools/offcputime.py -f -1
      usage: offcputime.py [-h] [-u | -p PID] [-v] [-f]
                           [--stack-storage-size STACK_STORAGE_SIZE]
                           [duration]
      offcputime.py: error: argument duration: must be positive and nonzero
      devbig680[bcc](no branch, rebasing abirchall_dev): ~/bcc_run_tool.sh offcputime --stack-storage-size 0 -f 1
      [Running] /data/users/abirchall/bcc/tools/offcputime.py --stack-storage-size 0 -f 1
      usage: offcputime.py [-h] [-u | -p PID] [-v] [-f]
                           [--stack-storage-size STACK_STORAGE_SIZE]
                           [duration]
      offcputime.py: error: argument --stack-storage-size: must be positive and nonzero
      devbig680[bcc](no branch, rebasing abirchall_dev): ~/bcc_run_tool.sh offcputime --stack-storage-size 1 -f 1
      [Running] /data/users/abirchall/bcc/tools/offcputime.py --stack-storage-size 1 -f 1
      swapper/27;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 7
      swapper/29;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 102
      swapper/25;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 113
      swapper/26;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 138
      swapper/28;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 164
      swapper/30;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 177
      swapper/24;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 980
      swapper/31;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 1556
      swapper/1;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2038
      swapper/3;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2190
      swapper/2;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2235
      swapper/15;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2260
      swapper/13;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2450
      swapper/6;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2646
      swapper/12;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2675
      swapper/11;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2769
      swapper/8;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2885
      swapper/4;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 3134
      swapper/10;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 3416
      swapper/14;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 3702
      swapper/7;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 4343
      swapper/9;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 5397
      swapper/5;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 6836
      swapper/17;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 13792
      swapper/19;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 14488
      swapper/22;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 15298
      swapper/18;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 15735
      swapper/20;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 17333
      swapper/16;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 17645
      swapper/23;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 17681
      swapper/21;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 18607
      WARNING: 92 stack traces could not be displayed. Consider increasing --stack-storage-size.
      47d871f0
    • Andrew Birchall's avatar
      handle ENOMEM in tools/offcputime · ee7e5b46
      Andrew Birchall authored
      Summary:
      BPF_STACK_TRACE(_name, _size) will allocate space for _size stack traces
      (see https://github.com/torvalds/linux/blob/master/kernel/bpf/stackmap.c#L30-L50).
      
      If we've already used all of this space, subsequent calls to bpf_get_stackid()
      will return -ENOMEM (see https://github.com/torvalds/linux/blob/master/kernel/bpf/stackmap.c#L173-L176).
      
      This causes our BPF bytecode to store this value in key_t.stack_id and
      subsequently causes our python application to crash due to a KeyError when
      invoking stack_traces.walk(k.stack_id).
      
      Let's avoid calling stack_traces.walk(k.stack_id) with back stackid's
      
      Test Plan:
      Run offcputime.py in an extreme case; with space for only a single stack trace
      ```
      devbig680[bcc](tools): sed_in_file 's/BPF_STACK_TRACE(stack_traces, 1024)/BPF_STACK_TRACE(stack_traces, 1)/' tools/offcputime.py && \
      > ~/bcc_run_tool.sh offcputime -f 5; \
      > git reset --hard HEAD
      swapper/30;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 496
      swapper/26;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 553
      swapper/28;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 604
      swapper/31;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 692
      swapper/23;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 713
      swapper/18;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 919
      swapper/16;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 1051
      swapper/20;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 1056
      swapper/21;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 1585
      swapper/24;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 1597
      swapper/27;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 1610
      swapper/17;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 1674
      swapper/22;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2390
      swapper/25;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2574
      swapper/19;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2589
      swapper/29;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 8428
      swapper/8;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 15272
      swapper/15;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 15591
      swapper/11;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 17934
      swapper/9;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 18100
      swapper/14;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 18266
      swapper/10;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 20124
      swapper/12;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 20887
      swapper/13;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 23453
      swapper/3;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 27296
      swapper/5;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 29094
      swapper/6;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 29799
      swapper/7;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 31522
      swapper/1;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 32269
      swapper/4;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 35585
      swapper/2;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 37627
      WARNING: 249 stack traces could not be displayed. Consider increasing stack trace storage size.
      HEAD is now at d3365e9 [RFC] handle ENOMEM in tools/offcputime`
      ```
      ee7e5b46
    • 4ast's avatar
      Merge pull request #542 from mbertrone/master · 50a74246
      4ast authored
      http_filter fixes
      50a74246
    • Bertrone Matteo's avatar