1. 24 May, 2016 1 commit
  2. 20 May, 2016 2 commits
  3. 17 May, 2016 3 commits
    • Brendan Gregg's avatar
      Merge pull request #543 from abirchall/abirchall_release · c7f317de
      Brendan Gregg authored
      add --kernel-threads-only to tools/offcputime
      c7f317de
    • Andrew Birchall's avatar
      add --kernel-threads-only to tools/offcputime · 582b5dd2
      Andrew Birchall authored
      Summary:
      Adds `--kernel-threads-only` arg
      The kernel-threads-only arg is exclusive with pid/user-threads-only via `parser.add_mutually_exclusive_group`.
      The output message now indicates what we are tracing (pid/user threads/kernel threads/all threads).
      Removed the --verbose arg (unused).
      
      Test Plan:
      Run with combinations of the args; validate output looks sane:
      
      // test mutually exclusive group
      ```
      devbig680[bcc](abirchall_dev): ~/bcc_run_tool.sh offcputime -k -u 1
      [Running] /data/users/abirchall/bcc/tools/offcputime.py -k -u 1
      usage: offcputime.py [-h] [-p PID | -k | -u] [-v] [-f] [duration]
      offcputime.py: error: argument -u/--user-threads-only: not allowed with argument -k/--kernel-threads-only
      ```
      
      // kernel threads only
      ```
      devbig680[bcc](abirchall_dev): ~/bcc_run_tool.sh offcputime -f -k 1
      [Running] /data/users/abirchall/bcc/tools/offcputime.py -f -k 1
      swapper/21;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 11
      swapper/16;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 19
      swapper/22;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 20
      swapper/31;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 20
      swapper/23;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 67
      swapper/25;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 74
      ...
      ```
      `~/bcc_run_tool.sh offcputime -f --kernel-threads-only 1`
      
      // user threads only
      `~/bcc_run_tool.sh offcputime -f --user-threads-only 1`
      `~/bcc_run_tool.sh offcputime -f -u 1`
      
      // specific pid
      `~/bcc_run_tool.sh offcputime -f -p $(pidof hphpi) 1`
      ```
      devbig680[bcc](abirchall_dev): ~/bcc_run_tool.sh offcputime --pid $(pidof mcrouter) 10 | head
      [Running] /data/users/abirchall/bcc/tools/offcputime.py --pid 95929 10
      Tracing off-CPU time (us) of PID 95929 by kernel stack for 10 secs.
      ```
      
      Note that this last case (specific PID) doesn't appear to be working; I can debug that in a follow up commit.
      582b5dd2
    • 4ast's avatar
      Merge pull request #533 from abirchall/abirchall_dev · 09534d89
      4ast authored
      handle ENOMEM in tools/offcputime
      09534d89
  4. 16 May, 2016 4 commits
    • Andrew Birchall's avatar
      move ENOMEM comment; create --stack-storage-size arg; misc cleanup · 47d871f0
      Andrew Birchall authored
      Summary:
      * move ENOMEM comment from tools/offcputime.py to src/cc/export/helpers.h
      * create --stack-storage-size arg to allow the user to set the stack storage size
          * requires a positive_nonzero_int type checking function for argparse
      * clean up the rest of the args a bit
          * use parser.add_mutually_exclusive_group instead of manually handling the
            mutual exclusion
          * use positive_nonzero_int for duration arg
          * use positive_int for pid arg
      * only print a warning about increasing the storage size if at least one of the
        get_stackid errors was a -ENOMEM
      * remove the debug param (we can add this manually when testing)
      * fix a bug where all processes are traced when specifying pid of 0
      
      Test Plan:
      ```
      devbig680[bcc](no branch, rebasing abirchall_dev): ~/bcc_run_tool.sh offcputime --help
      [Running] /data/users/abirchall/bcc/tools/offcputime.py --help
      usage: offcputime.py [-h] [-u | -p PID] [-v] [-f]
                           [--stack-storage-size STACK_STORAGE_SIZE]
                           [duration]
      
      Summarize off-CPU time by kernel stack trace
      
      positional arguments:
        duration              duration of trace, in seconds
      
      optional arguments:
        -h, --help            show this help message and exit
        -u, --useronly        user threads only (no kernel threads)
        -p PID, --pid PID     trace this PID only
        -v, --verbose         show raw addresses
        -f, --folded          output folded format
        --stack-storage-size STACK_STORAGE_SIZE
                              the number of unique stack traces that can be stored
                              and displayed
      
      examples:
          ./offcputime             # trace off-CPU stack time until Ctrl-C
          ./offcputime 5           # trace for 5 seconds only
          ./offcputime -f 5        # 5 seconds, and output in folded format
          ./offcputime -u          # don't include kernel threads (user only)
          ./offcputime -p 185      # trace fo PID 185 only
      devbig680[bcc](no branch, rebasing abirchall_dev): ~/bcc_run_tool.sh offcputime -f 0
      [Running] /data/users/abirchall/bcc/tools/offcputime.py -f 0
      usage: offcputime.py [-h] [-u | -p PID] [-v] [-f]
                           [--stack-storage-size STACK_STORAGE_SIZE]
                           [duration]
      offcputime.py: error: argument duration: must be positive and nonzero
      devbig680[bcc](no branch, rebasing abirchall_dev): ~/bcc_run_tool.sh offcputime -f -1
      [Running] /data/users/abirchall/bcc/tools/offcputime.py -f -1
      usage: offcputime.py [-h] [-u | -p PID] [-v] [-f]
                           [--stack-storage-size STACK_STORAGE_SIZE]
                           [duration]
      offcputime.py: error: argument duration: must be positive and nonzero
      devbig680[bcc](no branch, rebasing abirchall_dev): ~/bcc_run_tool.sh offcputime --stack-storage-size 0 -f 1
      [Running] /data/users/abirchall/bcc/tools/offcputime.py --stack-storage-size 0 -f 1
      usage: offcputime.py [-h] [-u | -p PID] [-v] [-f]
                           [--stack-storage-size STACK_STORAGE_SIZE]
                           [duration]
      offcputime.py: error: argument --stack-storage-size: must be positive and nonzero
      devbig680[bcc](no branch, rebasing abirchall_dev): ~/bcc_run_tool.sh offcputime --stack-storage-size 1 -f 1
      [Running] /data/users/abirchall/bcc/tools/offcputime.py --stack-storage-size 1 -f 1
      swapper/27;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 7
      swapper/29;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 102
      swapper/25;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 113
      swapper/26;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 138
      swapper/28;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 164
      swapper/30;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 177
      swapper/24;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 980
      swapper/31;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 1556
      swapper/1;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2038
      swapper/3;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2190
      swapper/2;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2235
      swapper/15;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2260
      swapper/13;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2450
      swapper/6;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2646
      swapper/12;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2675
      swapper/11;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2769
      swapper/8;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2885
      swapper/4;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 3134
      swapper/10;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 3416
      swapper/14;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 3702
      swapper/7;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 4343
      swapper/9;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 5397
      swapper/5;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 6836
      swapper/17;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 13792
      swapper/19;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 14488
      swapper/22;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 15298
      swapper/18;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 15735
      swapper/20;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 17333
      swapper/16;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 17645
      swapper/23;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 17681
      swapper/21;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 18607
      WARNING: 92 stack traces could not be displayed. Consider increasing --stack-storage-size.
      47d871f0
    • Andrew Birchall's avatar
      handle ENOMEM in tools/offcputime · ee7e5b46
      Andrew Birchall authored
      Summary:
      BPF_STACK_TRACE(_name, _size) will allocate space for _size stack traces
      (see https://github.com/torvalds/linux/blob/master/kernel/bpf/stackmap.c#L30-L50).
      
      If we've already used all of this space, subsequent calls to bpf_get_stackid()
      will return -ENOMEM (see https://github.com/torvalds/linux/blob/master/kernel/bpf/stackmap.c#L173-L176).
      
      This causes our BPF bytecode to store this value in key_t.stack_id and
      subsequently causes our python application to crash due to a KeyError when
      invoking stack_traces.walk(k.stack_id).
      
      Let's avoid calling stack_traces.walk(k.stack_id) with back stackid's
      
      Test Plan:
      Run offcputime.py in an extreme case; with space for only a single stack trace
      ```
      devbig680[bcc](tools): sed_in_file 's/BPF_STACK_TRACE(stack_traces, 1024)/BPF_STACK_TRACE(stack_traces, 1)/' tools/offcputime.py && \
      > ~/bcc_run_tool.sh offcputime -f 5; \
      > git reset --hard HEAD
      swapper/30;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 496
      swapper/26;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 553
      swapper/28;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 604
      swapper/31;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 692
      swapper/23;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 713
      swapper/18;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 919
      swapper/16;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 1051
      swapper/20;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 1056
      swapper/21;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 1585
      swapper/24;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 1597
      swapper/27;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 1610
      swapper/17;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 1674
      swapper/22;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2390
      swapper/25;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2574
      swapper/19;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 2589
      swapper/29;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 8428
      swapper/8;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 15272
      swapper/15;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 15591
      swapper/11;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 17934
      swapper/9;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 18100
      swapper/14;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 18266
      swapper/10;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 20124
      swapper/12;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 20887
      swapper/13;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 23453
      swapper/3;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 27296
      swapper/5;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 29094
      swapper/6;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 29799
      swapper/7;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 31522
      swapper/1;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 32269
      swapper/4;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 35585
      swapper/2;start_secondary;cpu_startup_entry;schedule_preempt_disabled;schedule 37627
      WARNING: 249 stack traces could not be displayed. Consider increasing stack trace storage size.
      HEAD is now at d3365e9 [RFC] handle ENOMEM in tools/offcputime`
      ```
      ee7e5b46
    • 4ast's avatar
      Merge pull request #542 from mbertrone/master · 50a74246
      4ast authored
      http_filter fixes
      50a74246
    • Bertrone Matteo's avatar
  5. 13 May, 2016 1 commit
  6. 12 May, 2016 3 commits
  7. 11 May, 2016 5 commits
  8. 06 May, 2016 13 commits
  9. 05 May, 2016 8 commits