1. 22 May, 2018 3 commits
    • Steve Wise's avatar
      RDMA/CMA: add rdma_iw_cm_id() and rdma_res_to_id() helpers · fbdb0a91
      Steve Wise authored
      Add a helper function for iwarp drivers to be able to map an
      rdma_cm_id to an iw_cm_id.  This is useful for dumping driver specific
      NLDEV/RESTRACK connection state.
      
      Add a helper to return the rdma_cm_id pointer from the rdma_restack
      pointer.  This is needed for rdma drivers to map a res entry back to
      the public rdma_cm_id struct.
      Signed-off-by: default avatarSteve Wise <swise@opengridcomputing.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      fbdb0a91
    • Steve Wise's avatar
      iw_cxgb4: always set iw_cm_id.provider_data · b06f2efd
      Steve Wise authored
      In active side connections, the provider_data field is not
      getting set.  This will be used in a subsequent patch to dump
      state, so always set it.
      Signed-off-by: default avatarSteve Wise <swise@opengridcomputing.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      b06f2efd
    • Doug Ledford's avatar
      RDMA/ipoib: Update paths on CLIENT_REREG/SM_CHANGE events · fa9391db
      Doug Ledford authored
      We do a light flush on CLIENT_REREG and SM_CHANGE events.  This goes
      through and marks paths invalid. But we weren't always checking for this
      validity when we needed to, and so we could keep using a path marked
      invalid.  What's more, once we establish a path with a valid ah, we put
      a pointer to the ah in the neigh struct directly, so even if we mark the
      path as invalid, as long as the neigh has a direct pointer to the ah, it
      keeps using the old, outdated ah.
      
      To fix this we do several things.
      
      1) Put the valid flag in the ah instead of the path struct, so when we
      put the ah pointer directly in the neigh struct, we can easily check the
      validity of the ah on send events.
      2) Check the neigh->ah and neigh->ah->valid elements in the needed
      places, and if we have an ah, but it's invalid, then invoke a refresh of
      the ah.
      3) Fix the various places that check for path, but didn't check for
      path->valid (now path->ah && path->ah->valid).
      Reported-by: default avatarEvgenii Smirnov <evgenii.smirnov@profitbricks.com>
      Fixes: ee1e2c82 ("IPoIB: Refresh paths instead of flushing them on SM change events")
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      fa9391db
  2. 17 May, 2018 10 commits
  3. 16 May, 2018 4 commits
  4. 15 May, 2018 8 commits
  5. 09 May, 2018 15 commits
    • Brian Welty's avatar
      IB/{hfi1, qib, rdmavt}: Move logic to allocate receive WQE into rdmavt · 832369fa
      Brian Welty authored
      Moving receive-side WQE allocation logic into rdmavt will allow
      further code reuse between qib and hfi1 drivers.
      Reviewed-by: default avatarMike Marciniszyn <mike.marciniszyn@intel.com>
      Reviewed-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
      Signed-off-by: default avatarBrian Welty <brian.welty@intel.com>
      Signed-off-by: default avatarHarish Chegondi <harish.chegondi@intel.com>
      Signed-off-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      832369fa
    • Sebastian Sanchez's avatar
      IB/{hfi1, rdmavt, qib}: Implement CQ completion vector support · 5d18ee67
      Sebastian Sanchez authored
      Currently the driver doesn't support completion vectors. These
      are used to indicate which sets of CQs should be grouped together
      into the same vector. A vector is a CQ processing thread that
      runs on a specific CPU.
      
      If an application has several CQs bound to different completion
      vectors, and each completion vector runs on different CPUs, then
      the completion queue workload is balanced. This helps scale as more
      nodes are used.
      
      Implement CQ completion vector support using a global workqueue
      where a CQ entry is queued to the CPU corresponding to the CQ's
      completion vector. Since the workqueue is global, it's guaranteed
      to always be there when queueing CQ entries; Therefore, the RCU
      locking for cq->rdi->worker in the hot path is superfluous.
      
      Each completion vector is assigned to a different CPU. The number of
      completion vectors available is computed by taking the number of
      online, physical CPUs from the local NUMA node and subtracting the
      CPUs used for kernel receive queues and the general interrupt.
      Special use cases:
      
        * If there are no CPUs left for completion vectors, the same CPU
          for the general interrupt is used; Therefore, there would only
          be one completion vector available.
      
        * For multi-HFI systems, the number of completion vectors available
          for each device is the total number of completion vectors in
          the local NUMA node divided by the number of devices in the same
          NUMA node. If there's a division remainder, the first device to
          get initialized gets an extra completion vector.
      
      Upon a CQ creation, an invalid completion vector could be specified.
      Handle it as follows:
      
        * If the completion vector is less than 0, set it to 0.
      
        * Set the completion vector to the result of the passed completion
          vector moded with the number of device completion vectors
          available.
      Reviewed-by: default avatarMike Marciniszyn <mike.marciniszyn@intel.com>
      Signed-off-by: default avatarSebastian Sanchez <sebastian.sanchez@intel.com>
      Signed-off-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      5d18ee67
    • Sebastian Sanchez's avatar
      IB/hfi1: Create common functions for affinity CPU mask operations · cf38ea10
      Sebastian Sanchez authored
      CPU masks are used to keep track of affinity assignments for IRQs
      and processes. Operations performed on these affinity CPU masks are
      duplicated throughout the code.
      
      Create common functions for affinity CPU mask operations to remove
      duplicate code.
      Reviewed-by: default avatarMichael J. Ruhl <michael.j.ruhl@intel.com>
      Signed-off-by: default avatarSebastian Sanchez <sebastian.sanchez@intel.com>
      Signed-off-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      cf38ea10
    • Sebastian Sanchez's avatar
      IB/hfi1: Optimize kthread pointer locking when queuing CQ entries · af8aab71
      Sebastian Sanchez authored
      All threads queuing CQ entries on different CQs are unnecessarily
      synchronized by a spin lock to check if the CQ kthread worker hasn't
      been destroyed before queuing an CQ entry.
      
      The lock used in 6efaf10f ("IB/rdmavt: Avoid queuing work into a
      destroyed cq kthread worker") is a device global lock and will have
      poor performance at scale as completions are entered from a large
      number of CPUs.
      
      Convert to use RCU where the read side of RCU is rvt_cq_enter() to
      determine that the worker is alive prior to triggering the
      completion event.
      Apply write side RCU semantics in rvt_driver_cq_init() and
      rvt_cq_exit().
      
      Fixes: 6efaf10f ("IB/rdmavt: Avoid queuing work into a destroyed cq kthread worker")
      Cc: <stable@vger.kernel.org> # 4.14.x
      Reviewed-by: default avatarMike Marciniszyn <mike.marciniszyn@intel.com>
      Signed-off-by: default avatarSebastian Sanchez <sebastian.sanchez@intel.com>
      Signed-off-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      af8aab71
    • Kamenee Arumugam's avatar
      IB/Hfi1: Read CCE Revision register to verify the device is responsive · c872a1f9
      Kamenee Arumugam authored
      When Hfi1 device is unresponsive, reading the RcvArrayCnt register
      will return all 1's. This value is then used to remap chip's RcvArray.
      The incorrect all ones value used in remapping RcvArray
      will cause warn on as shown by trace below:
      
      [<ffffffff81685eac>] dump_stack+0x19/0x1b
      [<ffffffff81085820>] warn_slowpath_common+0x70/0xb0
      [<ffffffff810858bc>] warn_slowpath_fmt+0x5c/0x80
      [<ffffffff81065c29>] __ioremap_caller+0x279/0x320
      [<ffffffff8142873c>] ? _dev_info+0x6c/0x90
      [<ffffffffa021d155>] ? hfi1_pcie_ddinit+0x1d5/0x330 [hfi1]
      [<ffffffff81065d62>] ioremap_wc+0x32/0x40
      [<ffffffffa021d155>] hfi1_pcie_ddinit+0x1d5/0x330 [hfi1]
      [<ffffffffa0204851>] hfi1_init_dd+0x1d1/0x2440 [hfi1]
      [<ffffffff813503dc>] ? pci_write_config_word+0x1c/0x20
      
      Read CCE revision register first to verify that WFR device is
      responsive. If the read return "all ones", bail out from init
      and fail the driver load.
      Reviewed-by: default avatarMike Marciniszyn <mike.marciniszyn@intel.com>
      Reviewed-by: default avatarMichael J. Ruhl <michael.j.ruhl@intel.com>
      Signed-off-by: default avatarKamenee Arumugam <kamenee.arumugam@intel.com>
      Signed-off-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      c872a1f9
    • Mitko Haralanov's avatar
      IB/hfi1: Rework fault injection machinery · a74d5307
      Mitko Haralanov authored
      The packet fault injection code present in the HFI1 driver had some
      issues which not only fragment the code but also created user
      confusion. Furthermore, it suffered from the following issues:
      
        1. The fault_packet method only worked for received packets. This
           meant that the only fault injection mode available for sent
           packets is fault_opcode, which did not allow for random packet
           drops on all egressing packets.
        2. The mask available for the fault_opcode mode did not really work
           due to the fact that the opcode values are not bits in a bitmask but
           rather sequential integer values. Creating a opcode/mask pair that
           would successfully capture a set of packets was nearly impossible.
        3. The code was fragmented and used too many debugfs entries to
           operate and control. This was confusing to users.
        4. It did not allow filtering fault injection on a per direction basis -
           egress vs. ingress.
      
      In order to improve or fix the above issues, the following changes have
      been made:
      
         1. The fault injection methods have been combined into a single fault
            injection facility. As such, the fault injection has been plugged
            into both the send and receive code paths. Regardless of method used
            the fault injection will operate on both egress and ingress packets.
         2. The type of fault injection - by packet or by opcode - is now controlled
            by changing the boolean value of the file "opcode_mode". When the value
            is set to True, fault injection is done by opcode. Otherwise, by
            packet.
         2. The masking ability has been removed in favor of a bitmap that holds
            opcodes of interest (one bit per opcode, a total of 256 bits). This
            works in tandem with the "opcode_mode" value. When the value of
            "opcode_mode" is False, this bitmap is ignored. When the value is
            True, the bitmap lists all opcodes to be considered for fault injection.
            By default, the bitmap is empty. When the user wants to filter by opcode,
            the user sets the corresponding bit in the bitmap by echo'ing the bit
            position into the 'opcodes' file. This gets around the issue that the set
            of opcodes does not lend itself to effective masks and allow for extremely
            fine-grained filtering by opcode.
         4. fault_packet and fault_opcode methods have been combined. Hence, there
            is only one debugfs directory controlling the entire operation of the
            fault injection machinery. This reduces the number of debugfs entries
            and provides a more unified user experience.
         5. A new control files - "direction" - is provided to allow the user to
            control the direction of packets, which are subject to fault injection.
         6. A new control file - "skip_usec" - is added that would allow the user
            to specify a "timeout" during which no fault injection will occur.
      
      In addition, the following bug fixes have been applied:
      
         1. The fault injection code has been split into its own header and source
            files. This was done to better organize the code and support conditional
            compilation without littering the code with #ifdef's.
         2. The method by which the TX PIO packets were being marked for drop
            conflicted with the way send contexts were being setup. As a result,
            the send context was repeatedly being reset.
         3. The fault injection only makes sense when the user can control it
            through the debugfs entries. However, a kernel configuration can
            enable fault injection but keep fault injection debugfs entries
            disabled. Therefore, it makes sense that the HFI fault injection
            code depends on both.
         4. Error suppression did not take into account the method by which PIO
            packets were being dropped. Therefore, even with error suppression
            turned on, errors would still be displayed to the screen. A larger
            enough packet drop percentage would case the kernel to crash because
            the driver would be stuck printing errors.
      Reviewed-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
      Reviewed-by: default avatarDon Hiatt <don.hiatt@intel.com>
      Reviewed-by: default avatarMike Marciniszyn <mike.marciniszyn@intel.com>
      Signed-off-by: default avatarMitko Haralanov <mitko.haralanov@intel.com>
      Signed-off-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      a74d5307
    • Alex Estrin's avatar
      IB/{hfi1, qib}: Add handling of kernel restart · 8d3e7113
      Alex Estrin authored
      A warm restart will fail to unload the driver, leaving link state
      potentially flapping up to the point the BIOS resets the adapter.
      Correct the issue by hooking the shutdown pci method,
      which will bring port down.
      
      Cc: <stable@vger.kernel.org> # 4.9.x
      Reviewed-by: default avatarMike Marciniszyn <mike.marciniszyn@intel.com>
      Signed-off-by: default avatarAlex Estrin <alex.estrin@intel.com>
      Signed-off-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      8d3e7113
    • Michael J. Ruhl's avatar
      IB/hfi1: Reorder incorrect send context disable · a93a0a31
      Michael J. Ruhl authored
      User send context integrity bits are cleared before the context is
      disabled.  If the send context is still processing data, any packets
      that need those integrity bits will cause an error and halt the send
      context.
      
      During the disable handling, the driver waits for the context to drain.
      If the context is halted, the driver will eventually timeout because
      the context won't drain and then incorrectly bounce the link.
      
      Reorder the bit clearing and the context disable.
      
      Examine the software state and send context status as well as the
      egress status to determine if a send context is in the halted state.
      
      Promote the check macros to static functions for consistency with the
      new check and to follow kernel style.
      
      Remove an unused define that refers to the egress timeout.
      
      Cc: <stable@vger.kernel.org> # 4.9.x
      Reviewed-by: default avatarMitko Haralanov <mitko.haralanov@intel.com>
      Reviewed-by: default avatarMike Marciniszyn <mike.marciniszyn@intel.com>
      Signed-off-by: default avatarMichael J. Ruhl <michael.j.ruhl@intel.com>
      Signed-off-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      a93a0a31
    • Michael J. Ruhl's avatar
      IB/hfi1: Return correct value for device state · e4607073
      Michael J. Ruhl authored
      The driver_pstate() function is used to map internal driver state
      information to externally defined states.
      
      The VERIFY_CAP and GOING_UP states are config/training states, but
      the mapping routing returns the POLLING value.
      
      Update the return values for VERIFY_CAP and GOING_UP to return the
      correct value: TRAINING.
      Reviewed-by: default avatarSebastian Sanchez <sebastian.sanchez@intel.com>
      Signed-off-by: default avatarMichael J. Ruhl <michael.j.ruhl@intel.com>
      Signed-off-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      e4607073
    • Mike Marciniszyn's avatar
      IB/hfi1: Fix fault injection init/exit issues · 8c79d822
      Mike Marciniszyn authored
      There are config dependent code paths that expose panics in unload
      paths both in this file and in debugfs_remove_recursive() because
      CONFIG_FAULT_INJECTION and CONFIG_FAULT_INJECTION_DEBUG_FS can be
      set independently.
      
      Having CONFIG_FAULT_INJECTION set and CONFIG_FAULT_INJECTION_DEBUG_FS
      reset causes fault_create_debugfs_attr() to return an error.
      
      The debugfs.c routines tolerate failures, but the module unload panics
      dereferencing a NULL in the two exit routines.  If that is fixed, the
      dir passed to debugfs_remove_recursive comes from a memory location
      that was freed and potentially reused causing a segfault or corrupting
      memory.
      
      Here is an example of the NULL deref panic:
      
      [66866.286829] BUG: unable to handle kernel NULL pointer dereference at 0000000000000088
      [66866.295602] IP: hfi1_dbg_ibdev_exit+0x2a/0x80 [hfi1]
      [66866.301138] PGD 858496067 P4D 858496067 PUD 8433a7067 PMD 0
      [66866.307452] Oops: 0000 [#1] SMP
      [66866.310953] Modules linked in: hfi1(-) rdmavt rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm iw_cm ib_cm ib_core rpcsec_gss_krb5 nfsv4 dns_resolver nfsv3 nfs fscache sb_edac x86_pkg_temp_thermal intel_powerclamp vfat fat coretemp kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc aesni_intel iTCO_wdt iTCO_vendor_support crypto_simd mei_me glue_helper cryptd mxm_wmi ipmi_si pcspkr lpc_ich sg mei ioatdma ipmi_devintf i2c_i801 mfd_core shpchp ipmi_msghandler wmi acpi_power_meter acpi_cpufreq nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables ext4 mbcache jbd2 sd_mod mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt igb fb_sys_fops ttm ahci ptp crc32c_intel libahci pps_core drm dca libata i2c_algo_bit i2c_core [last unloaded: opa_vnic]
      [66866.385551] CPU: 8 PID: 7470 Comm: rmmod Not tainted 4.14.0-mam-tid-rdma #2
      [66866.393317] Hardware name: Intel Corporation S2600WT2/S2600WT2, BIOS SE5C610.86B.01.01.0018.C4.072020161249 07/20/2016
      [66866.405252] task: ffff88084f28c380 task.stack: ffffc90008454000
      [66866.411866] RIP: 0010:hfi1_dbg_ibdev_exit+0x2a/0x80 [hfi1]
      [66866.417984] RSP: 0018:ffffc90008457da0 EFLAGS: 00010202
      [66866.423812] RAX: 0000000000000000 RBX: ffff880857de0000 RCX: 0000000180040001
      [66866.431773] RDX: 0000000180040002 RSI: ffffea0021088200 RDI: 0000000040000000
      [66866.439734] RBP: ffffc90008457da8 R08: ffff88084220e000 R09: 0000000180040001
      [66866.447696] R10: 000000004220e001 R11: ffff88084220e000 R12: ffff88085a31c000
      [66866.455657] R13: ffffffffa07c9820 R14: ffffffffa07c9890 R15: ffff881059d78100
      [66866.463618] FS:  00007f6876047740(0000) GS:ffff88085f800000(0000) knlGS:0000000000000000
      [66866.472644] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
      [66866.479053] CR2: 0000000000000088 CR3: 0000000856357006 CR4: 00000000001606e0
      [66866.487013] Call Trace:
      [66866.489747]  remove_one+0x1f/0x220 [hfi1]
      [66866.494221]  pci_device_remove+0x39/0xc0
      [66866.498596]  device_release_driver_internal+0x141/0x210
      [66866.504424]  driver_detach+0x3f/0x80
      [66866.508409]  bus_remove_driver+0x55/0xd0
      [66866.512784]  driver_unregister+0x2c/0x50
      [66866.517164]  pci_unregister_driver+0x2a/0xa0
      [66866.521934]  hfi1_mod_cleanup+0x10/0xaa2 [hfi1]
      [66866.526988]  SyS_delete_module+0x171/0x250
      [66866.531558]  do_syscall_64+0x67/0x1b0
      [66866.535644]  entry_SYSCALL64_slow_path+0x25/0x25
      [66866.540792] RIP: 0033:0x7f6875525c27
      [66866.544777] RSP: 002b:00007ffd48528e78 EFLAGS: 00000206 ORIG_RAX: 00000000000000b0
      [66866.553224] RAX: ffffffffffffffda RBX: 0000000001cc01d0 RCX: 00007f6875525c27
      [66866.561185] RDX: 00007f6875596000 RSI: 0000000000000800 RDI: 0000000001cc0238
      [66866.569146] RBP: 0000000000000000 R08: 00007f68757e9060 R09: 00007f6875596000
      [66866.577120] R10: 00007ffd48528c00 R11: 0000000000000206 R12: 00007ffd48529db4
      [66866.585080] R13: 0000000000000000 R14: 0000000001cc01d0 R15: 0000000001cc0010
      [66866.593040] Code: 90 0f 1f 44 00 00 48 83 3d a3 8b 03 00 00 55 48 89 e5 53 48 89 fb 74 4e 48 8d bf 18 0c 00 00 e8 9d f2 ff ff 48 8b 83 20 0c 00 00 <48> 8b b8 88 00 00 00 e8 2a 21 b3 e0 48 8b bb 20 0c 00 00 e8 0e
      [66866.614127] RIP: hfi1_dbg_ibdev_exit+0x2a/0x80 [hfi1] RSP: ffffc90008457da0
      [66866.621885] CR2: 0000000000000088
      [66866.625618] ---[ end trace c4817425783fb092 ]---
      
      Fix by insuring that upon failure from fault_create_debugfs_attr() the
      parent pointer for the routines is always set to NULL and guards added
      in the exit routines to insure that debugfs_remove_recursive() is not
      called when when the parent pointer is NULL.
      
      Fixes: 0181ce31 ("IB/hfi1: Add receive fault injection feature")
      Cc: <stable@vger.kernel.org> # 4.14.x
      Reviewed-by: default avatarMichael J. Ruhl <michael.j.ruhl@intel.com>
      Signed-off-by: default avatarMike Marciniszyn <mike.marciniszyn@intel.com>
      Signed-off-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      8c79d822
    • Alex Estrin's avatar
      IB/hfi1: Complete check for locally terminated smp · 959f2d17
      Alex Estrin authored
      For lid routed packets 'hop_cnt' is zero, therefore current
      test is incomplete. Fix it by using local mad check for
      both lid routed and direct routed MADs.
      Reviewed-by: default avatarMike Mariciniszyn <mike.marciniszyn@intel.com>
      Signed-off-by: default avatarAlex Estrin <alex.estrin@intel.com>
      Signed-off-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      959f2d17
    • Michael J. Ruhl's avatar
      IB/hfi1: Return actual error value from program_rcvarray() · 48e0a655
      Michael J. Ruhl authored
      A failure of program_rcvarray() is treated inconsistently by the
      calling function.  In one case the error is returned, in a second
      case, the error is overwritten with EFAULT.  In both cases the
      code path is doing the same thing, allocating memory for groups,
      so it should be consistent.
      
      Make the error path consistent and return the error generated by
      program_rcvarray().
      Reviewed-by: default avatarHarish Chegondi <harish.chegondi@intel.com>
      Fixes: 7e7a436e ("staging/hfi1: Add TID entry program function body")
      Signed-off-by: default avatarMichael J. Ruhl <michael.j.ruhl@intel.com>
      Signed-off-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      48e0a655
    • Sebastian Sanchez's avatar
      IB/hfi1: Prevent LNI hang when LCB can't obtain lanes · 254361c1
      Sebastian Sanchez authored
      When the LCB isn't able to get any lanes operational on the
      first transition into mission mode, the link transfer active
      never happens and the LNI stays in the polling state indefinitely.
      
      Reset LCB upon receiving an 8051 interrupt for LCB to try to obtain
      lanes with firmware version 1.25.0 or later. Also, update the LCB
      reset value in other parts of the code with a macro defined to make
      the code more maintainable and rename functions with the link_width
      label to link_mode to reflect the fact that those functions set and
      read link related data not just the link width.
      Reviewed-by: default avatarMichael J. Ruhl <michael.j.ruhl@intel.com>
      Reviewed-by: default avatarMike Marciniszyn <mike.marciniszyn@intel.com>
      Signed-off-by: default avatarSebastian Sanchez <sebastian.sanchez@intel.com>
      Signed-off-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      254361c1
    • Doug Ledford's avatar
      Merge branch 'k.o/for-rc' into k.o/wip/dl-for-next · f5e27a20
      Doug Ledford authored
      Several items of conflict have arisen between the RDMA stack's for-rc
      branch and upcoming for-next work:
      
      9fd4350b ("IB/rxe: avoid double kfree_skb") directly conflicts with
      2e473507 ("IB/rxe: optimize the function duplicate_request")
      
      Patches already submitted by Intel for the hfi1 driver will fail to
      apply cleanly without this merge
      
      Other people on the mailing list have notified that their upcoming
      patches also fail to apply cleanly without this merge
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      f5e27a20
    • Idan Burstein's avatar
      IB/mlx5: posting klm/mtt list inline in the send queue for reg_wr · 064e5262
      Idan Burstein authored
      As most kernel RDMA ULPs, (e.g. NVMe over Fabrics in its default
      "register_always=Y" mode) registers and invalidates user buffer
      upon each IO.
      
      Today the mlx5 driver is posting the registration work
      request using scatter/gather entry for the MTT/KLM list.
      The fetch of the MTT/KLM list becomes the bottleneck in
      number of IO operation could be done by NVMe over Fabrics
      host driver on a single adapter as shown below.
      
      This patch is adding the support for inline registration
      work request upon MTT/KLM list of size <=64B.
      
      The result for NVMe over Fabrics is increase of > x3.5 for small
      IOs as shown below, I expect other ULPs (e.g iSER, SRP, NFS over RDMA)
      performance to be enhanced as well.
      
      The following results were taken against a single NVMe-oF (RoCE link layer)
      subsystem with a single namespace backed by null_blk using fio benchmark
      (with rw=randread, numjobs=48, iodepth={16,64}, ioengine=libaio direct=1):
      
      ConnectX-5 (pci Width x16)
      ---------------------------
      
      Block Size       s/g reg_wr            inline reg_wr
      ++++++++++     +++++++++++++++        ++++++++++++++++
      512B            1302.8K/34.82%         4951.9K/99.02%
      1KB             1284.3K/33.86%         4232.7K/98.09%
      2KB             1238.6K/34.1%          2797.5K/80.04%
      4KB             1169.3K/32.46%         1941.3K/61.35%
      8KB             1013.4K/30.08%         1236.6K/39.47%
      16KB            695.7K/20.19%          696.9K/20.59%
      32KB            350.3K/9.64%           350.6K/10.3%
      64KB            175.86K/5.27%          175.9K/5.28%
      
      ConnectX-4 (pci Width x8)
      ---------------------------
      
      Block Size       s/g reg_wr            inline reg_wr
      ++++++++++     +++++++++++++++        ++++++++++++++++
      512B            1285.8K/42.66%          4242.7K/98.18%
      1KB             1254.1K/41.74%          3569.2K/96.00%
      2KB             1185.9K/39.83%          2173.9K/75.58%
      4KB             1069.4K/36.46%          1343.3K/47.47%
      8KB             755.1K/27.77%           748.7K/29.14%
      Tested-by: default avatarNitzan Carmi <nitzanc@mellanox.com>
      Signed-off-by: default avatarIdan Burstein <idanb@mellanox.com>
      Signed-off-by: default avatarMax Gurtovoy <maxg@mellanox.com>
      Signed-off-by: default avatarLeon Romanovsky <leonro@mellanox.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      064e5262