• Chuck Lever's avatar
    svcrdma: Eliminate allocation of recv_ctxt objects in backchannel · b541dd55
    Chuck Lever authored
    The svc_rdma_recv_ctxt free list uses a lockless list to avoid the
    need for a spin lock in the fast path. llist_del_first(), which is
    used by svc_rdma_recv_ctxt_get(), requires serialization, however,
    when there are multiple list producers that are unserialized.
    
    I mistakenly thought there was only one caller of
    svc_rdma_recv_ctxt_get() (svc_rdma_refresh_recvs()), thus explicit
    serialization would not be necessary. But there is another caller:
    svc_rdma_bc_sendto(), and these two are not serialized against each
    other. I haven't seen ill effects that I could directly ascribe to
    a lack of serialization. It's just an observation based on code
    audit.
    
    When DMA-mapping before sending a Reply, the passed-in struct
    svc_rdma_recv_ctxt is used only for its write and reply PCLs. These
    are currently always empty in the backchannel case. So, instead of
    passing a full svc_rdma_recv_ctxt object to
    svc_rdma_map_reply_msg(), let's pass in just the Write and Reply
    PCLs.
    
    This change makes it unnecessary for the backchannel to acquire a
    dummy svc_rdma_recv_ctxt object when sending an RPC Call. The need
    for svc_rdma_recv_ctxt free list serialization is now completely
    avoided.
    Signed-off-by: default avatarChuck Lever <chuck.lever@oracle.com>
    b541dd55
svc_rdma_sendto.c 30.5 KB