• Hariprasad Shenai's avatar
    cxgb4/iw_cxgb4: work request logging feature · 7730b4c7
    Hariprasad Shenai authored
    This commit enhances the iwarp driver to optionally keep a log of rdma
    work request timining data for kernel mode QPs.  If iw_cxgb4 module option
    c4iw_wr_log is set to non-zero, each work request is tracked and timing
    data maintained in a rolling log that is 4096 entries deep by default.
    Module option c4iw_wr_log_size_order allows specifing a log2 size to use
    instead of the default order of 12 (4096 entries). Both module options
    are read-only and must be passed in at module load time to set them. IE:
    
    modprobe iw_cxgb4 c4iw_wr_log=1 c4iw_wr_log_size_order=10
    
    The timing data is viewable via the iw_cxgb4 debugfs file "wr_log".
    Writing anything to this file will clear all the timing data.
    Data tracked includes:
    
    - The host time when the work request was posted, just before ringing
    the doorbell.  The host time when the completion was polled by the
    application.  This is also the time the log entry is created.  The delta
    of these two times is the amount of time took processing the work request.
    
    - The qid of the EQ used to post the work request.
    
    - The work request opcode.
    
    - The cqe wr_id field.  For sq completions requests this is the swsqe
    index.  For recv completions this is the MSN of the ingress SEND.
    This value can be used to match log entries from this log with firmware
    flowc event entries.
    
    - The sge timestamp value just before ringing the doorbell when
    posting,  the sge timestamp value just after polling the completion,
    and CQE.timestamp field from the completion itself.  With these three
    timestamps we can track the latency from post to poll, and the amount
    of time the completion resided in the CQ before being reaped by the
    application.  With debug firmware, the sge timestamp is also logged by
    firmware in its flowc history so that we can compute the latency from
    posting the work request until the firmware sees it.
    Signed-off-by: default avatarSteve Wise <swise@opengridcomputing.com>
    Signed-off-by: default avatarHariprasad Shenai <hariprasad@chelsio.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    7730b4c7
iw_cxgb4.h 25.8 KB