• Yonghong Song's avatar
    fix hang with "trace.py --max-events # · a7554fc4
    Yonghong Song authored
    Currently running "trace.py --max-events #" (shortname "trace.py -M #")
    will hang like below:
    
      -bash-4.2$ sudo ./trace.py -M 2 'SyS_futex'
      PID     TID     COMM            FUNC
      137727  138229  IOThreadPool0   SyS_futex
      138250  151288  OBC Dispatcher  SyS_futex
      ^C^Z
      [1]+  Stopped                 sudo ./trace.py -M 2 'SyS_futex'
      -bash-4.2$
    
    The hang happens in perf_reader_free.
    
    Commit cd5d4a6c ("fix a race condition between perf_reader
    munmap and read") fixed a race condition by adding some coordination
    between perf_reader_event_read and perf_reader_free.
    
    In this case, however, the callback function inside perf_reader_event_read
    never returns and actually calling exit(). This is because
    the maximum number of perf_events have been received. The exit()
    is calling BPF object cleanup() which calls perf_reader_free to
    free the ring buffer. perf_reader_free got stuck since it thinks
    perf_reader_event_reader did not finish yet.
    
    To fix this, a checking for thread_id is added so that
    perf_reader_free will proceed without locking if the perf_reader_read
    tid is the same as perf_reader_free since this signals that
    the callback function calls/triggers perf_reader_free.
    
    After the fix,
    
      -bash-4.2$ sudo ./trace.py -M 2 'SyS_futex'
      PID     TID     COMM            FUNC
      137727  138178  load-monitor    SyS_futex
      6359    6440    load-monitor    SyS_futex
      -bash-4.2$
    Reported-by: default avatarTeng Qin <qinteng@fb.com>
    Signed-off-by: default avatarYonghong Song <yhs@fb.com>
    a7554fc4
perf_reader.c 9.05 KB