- 20 Aug, 2019 40 commits
-
-
James Smart authored
Typical SLI-4 hardware supports up to 2 4KB pages to be registered per XRI to contain the exchanges Scatter/Gather List. This caps the number of SGL elements that can be in the SGL. There are not extensions to extend the list out of the 2 pages. The G7 hardware adds a SGE type that allows the SGL to be vectored to a different scatter/gather list segment. And that segment can contain a SGE to go to another segment and so on. The initial segment must still be pre-registered for the XRI, but it can be a much smaller amount (256Bytes) as it can now be dynamically grown. This much smaller allocation can handle the SG list for most normal I/O, and the dynamic aspect allows it to support many MB's if needed. The implementation creates a pool which contains "segments" and which is initially sized to hold the initial small segment per xri. If an I/O requires additional segments, they are allocated from the pool. If the pool has no more segments, the pool is grown based on what is now needed. After the I/O completes, the additional segments are returned to the pool for use by other I/Os. Once allocated, the additional segments are not released under the assumption of "if needed once, it will be needed again". Pools are kept on a per-hardware queue basis, which is typically 1:1 per cpu, but may be shared by multiple cpus. The switch to the smaller initial allocation significantly reduces the memory footprint of the driver (which only grows if large ios are issued). Based on the several K of XRIs for the adapter, the 8KB->256B reduction can conserve 32MBs or more. It has been observed with per-cpu resource pools that allocating a resource on CPU A, may be put back on CPU B. While the get routines are distributed evenly, only a limited subset of CPUs may be handling the put routines. This can put a strain on the lpfc_put_cmd_rsp_buf_per_cpu routine because all the resources are being put on a limited subset of CPUs. Signed-off-by:
Dick Kennedy <dick.kennedy@broadcom.com> Signed-off-by:
James Smart <jsmart2021@gmail.com> Signed-off-by:
Martin K. Petersen <martin.petersen@oracle.com>
-
James Smart authored
Added code to support driver loopback with MDS Diagnostics. This style of diagnostics passes frames from the fabric to the driver who then echo them back out the link. SEND_FRAME WQEs are used to transmit the frames. Added the SOF and EOF field location definitions for use by SEND_FRAME. Also ensure that enable_mds_diags is a RW parameter. Signed-off-by:
-
James Smart authored
To aid better hardware detection when there are issues, report the first and second level hardware revisions from the READ_REV command. Add the elements to the existing hardware id string. Signed-off-by:
-
James Smart authored
In order to see real addresses, convert %p with %px for kernel addresses and replace %p with %pf for functions. While converting, standardize on "x%px" throughout (not %px or 0x%px). Signed-off-by:
-
James Smart authored
While performing code review, several relatively simple optimizations can be done in the fast path. Add these optimizations (unlikely designators). Signed-off-by:
-
James Smart authored
Running on Coverity produced the following errors: - coding style (indentation) - memset size mismatch errors note: comment cases where it is purposely a mismatch Fix the errors. Signed-off-by:
-
James Smart authored
modinfo for lpfc_nvme_enable_fb is incorrect. FirstBurst on lpfc target is not fully supported. Update the attribute description Signed-off-by:
-
James Smart authored
The driver is allowing the user to change lpfc_enable_bg while loading the driver against a FCoE adapter. This is not supported. No check is made for the adapter type when applying the blockguard enablement value. Fix by verifying the adapter type before setting the enablement flag. Signed-off-by:
-
James Smart authored
GetTrunkInfo is displaying an incorrect link speed when the link is a trunk and the link has gone down. The driver is not clearing the logical speed as part of the link down transition. Fix by setting the logical speed to UNKNOWN SPEED when the link goes down. Signed-off-by:
-
James Smart authored
Max Frame Size value is shown as 34816 in fdmishow from Switch. The driver uses bbRcvSize in common service param which is obtained from the READ_SPARM mailbox command. The bbRcvSize field which is displayed is a three nibble field but the driver is printing a full four nibbles. Fix by masking off the upper nibble. Signed-off-by:
-
James Smart authored
The scsi transport fc bsg interface does not expect the bsg_job_done() callback to be done if the bsg request call returns failure. Several of the HST_VENDOR cases in the driver unconditionally call bsg_job_done() regardless of the returning value. Fix the code to only call bsg_job_done() if the call to lpfc_bsg_request() will return success. Signed-off-by:
-
James Smart authored
When forcing the use of MSI (vs MSI-X) the driver is crashing in pci_irq_get_affinity. The driver was not using the new pci_alloc_irq_vectors interface in the MSI path. Fix by using pci_alloc_irq_vectors() with PCI_RQ_MSI in the MSI path. Signed-off-by:
-
James Smart authored
The driver is currently reporting a non-zero nvme sg_seg_cnt value of 256 when nvme is disabled. It should be zero. Fix by ensuring the value is cleared. Signed-off-by:
-
James Smart authored
If an unsolicited ABTS was received, the driver looks up the exchange it references. It it does various searches looking for the exchange context. When one is eventually matched and it is associated with an XRI context, the driver sends an ABORT WQE to terminate the exchange. Current code looks at whether the transport had taken action on the XRI yet or not (no action if set to LPFC_NVMET_STE_RCV; action if non-LPFC_NVMET_STE_RCV). Based on action or not one of two (sol vs unsol) issue abort routines are called. The unsol version cheats and transmits a sequence containing an ABTS with no interaction with the adapter. The sol version issues an Abort WQE and lets the adapter manage whether the ABTS is sent to not. The issue is the unsol version is sending ABTS unconditionally for the exchange that received the ABTS. It's unnecessary. Remove the conditional and just call the adapter command-based routine to let the adapter manage the ABTS. Signed-off-by:
-
James Smart authored
As part of firmware download, the adapter is reset. On the adapter the reset causes the function to stop and all outstanding io is terminated (without responses). The reset path then starts teardown of the adapter, starting with deregistration of the remote ports with the nvme-fc transport. The local port is then deregistered and the driver waits for local port deregistration. This never finishes. The remote port deregistrations terminated the nvme controllers, causing them to send aborts for all the outstanding io. The aborts were serviced in the driver, but stalled due to its state. The nvme layer then stops to reclaim it's outstanding io before continuing. The io must be returned before the reset on the controller is deemed complete and the controller delete performed. The remote port deregistration won't complete until all the controllers are terminated. And the local port deregistration won't complete until all controllers and remote ports are terminated. Thus things hang. The issue is the reset which stopped the adapter also stopped all the responses that would drive i/o completions, and the aborts were also stopped that stopped i/o completions. The driver, when resetting the adapter like this, needs to be generating the completions as part of the adapter reset so that I/O complete (in error), and any aborts are not queued. Fix by adding flush routines whenever the adapter port has been reset or discovered in error. The flush routines will generate the completions for the scsi and nvme outstanding io. The abort ios, if waiting, will be caught and flushed as well. Signed-off-by:
-
James Smart authored
This issue is specific to SLI-3 adapters, specifically when DIF is used. Once seen, this message floods the logs: 9064 BLKGRD: lpfc_scsi_prep_dma_buf_s3: Too many sg segments from dma_map_sg The driver, upon detecting an error such as too many elements in an sglist, misrepresents the error by treating it as a temporary resource issue by returning MLQUEUE_HOST_BUSY. In these cases, no retry will fix it and it should have been a hard error. The repeated retry was causing the spamming of the log. As for the initial reason of why an I/O encountered this issue at all is not clear as parameters set by the driver should have avoided this. The dm multipath maintainer has been notified of the issue. Fix by changing the return code for the dma mapping routines to indicate cases that are not retryable and return DID_ERROR on those cases. Signed-off-by:
-
James Smart authored
If the adapter encounters a condition which causes the adapter to fail (driver must detect the failure) simultaneously to a request to the driver to reset the adapter (such as a host_reset), the reset path will be racing with the asynchronously-detect adapter failure path. In the failing situation, one path has started to tear down the adapter data structures (io_wq's) while the other path has initiated a repeat of the teardown and is in the lpfc_sli_flush_xxx_rings path and attempting to access the just-freed data structures. Fix by the following: - In cases where an adapter failure is detected, rather than explicitly calling offline_eratt() to start the teardown, change the adapter state and let the later calls of posted work to the slowpath thread invoke the adapter recovery. In essence, this means all requests to reset are serialized on the slowpath thread. - Clean up the routine that restarts the adapter. If there is a failure from brdreset, don't immediately error and leave things in a partial state. Instead, ensure the adapter state is set and finish the teardown of structures before returning. - If in the scsi host reset handler and the board fails to reset and restart (which can be due to parallel reset/recovery paths), instead of hard failing and explicitly calling offline_eratt() (which gets into the redundant path), just fail out and let the asynchronous path resolve the adapter state. Signed-off-by:
-
James Smart authored
During cable pull testing a deadlock was seen between lpfc_nlp_counters() vs lpfc_mbox_process_link_up() vs lpfc_work_list_done(). They are all waiting on the shost->host_lock. Issue is all of these cases raise irq when taking out the lock but use spin_unlock_irq() when unlocking. The unlock path is will unconditionally re-enable interrupts in cases where irq state should be preserved. The re-enablement allowed the other paths to execute which then causes the deadlock. Fix by converting the lock/unlock to irqsave/irqrestore. Signed-off-by:
-
James Smart authored
In a test with high nvme remote port counts connected via a multi-hop FC switch config where switches were systematically reset (e.g. fabric partitioning and re-establishment), the nvme remote ports would switch addresses based on the switch reconfiguration events. The driver would get into a situation where the nvme port changed address, PLOGI and PRLI would succeed nvme transport registration occurred, but subsequent LS requests by the nvme subsystem failed due to a bad ndlp state and connectivity to the device failed. The driver hit a race condition on multiple devices that address swapped simultaneously. In cases where the driver notices the remote port structure came back as the same value as previously (meaning a nvme_rport structure was re-enabled and did not go through devloss_tmo/connect_tmo_failures on all controllers) the driver would unconditionally exit assuming the ndlp information was correct. But, if the ndlp's had been swapped, the ndlp had stale port state information, which when used by the LS request commands, would fail the commands. Fix by checking whether a node swap had occurred, and only exit if no ndlp swap had occurred. Signed-off-by:
-
James Smart authored
In situations where zoning is not being used, thus NVMe initiators see other NVMe initiators as well as NVMe targets, a link bounce on an initiator will cause the NVMe initiators to spew "6169" State Error messages. The driver is not qualifying whether the remote port is a NVMe targer or not before calling the lpfc_nvme_rescan_port(), which validates the role and prints the message if its only an NVMe initiator. Fix by the following: - Before calling lpfc_nvme_rescan_port() ensure that the node is a NVMe storage target or a NVMe discovery controller. - Clean up implementation of lpfc_nvme_rescan_port. remoteport pointer will always be NULL if a NVMe initiator only. But, grabbing of remoteport pointer should be done under lock to coincide with the registering of the remote port with the fc transport. Signed-off-by:
-
James Smart authored
On an SLI-3 adapter which does not support NVMe, but with the driver global attribute to enable nvme on any adapter if it does support NVMe (e.g. module parameter lpfc_enable_fc4_type=3), the SGL and total SGE values are being munged by the protocol enablement when it shouldn't be. Correct by changing the location of where the NVME sgl information is being applied, which will avoid any SLI-3-based adapter. Signed-off-by:
-
James Smart authored
If admin changes the devloss_tmo on an rport via the fc_remote_port rport dev_loss_tmo attribute, the value is on set on scsi stack. The change is not propagated to NVMe. The set routine in the lldd lacks the call to nvme_fc_set_remoteport_devloss() to set the value. Fix by adding the call to the lldd set routine. Signed-off-by:
-
James Smart authored
In tests with remote ports contantly logging out/logging coupled with occassional local link bounce, if a remote port is disocnnected for longer than devloss_tmo and then subsequently reconnected, eventually the test will fail to login with the remote port and remote port connectivity is lost. When devloss_tmo expires, the driver does not free the node struct until the port or npiv instances is being deleted. The node is left allocated but the state set to UNUSED. If the node was in the process of logging in when the local link drop occurred, meaning the RPI was allocated for the node in order to send the ELS, but not yet registered which comes after successful login, the node is moved to the NPR state, and if devloss expires, to UNUSED state. If the remote port comes back, the node associated with it is restarted and this path happens to allocate a new RPI and overwrites the prior RPI value. In the cases where the port was logged in and loggs out, the path did release the RPI but did not set the node rpi value. In the cases where the remote port never finished logging in, the path never did the call to release the rpi. In this latter case, when the node is subsequently restore, the new rpi allocation overwrites the rpi that was not released, and the rpi is now leaked. Eventually the port will run out of RPI resources to log into new remote ports. Fix by following changes: - When an rpi is released, do so under locks and ensure the node rpi value is set to a non-allocated value (LPFC_RPI_ALLOC_ERROR). Note: refactored to a small service routine to avoid indentation issues. - When re-enabling a node, check the rpi value to determine if a new allocation is necessary. If already set, use the prior rpi. Enhanced logging to help in the future. Signed-off-by:
-
James Smart authored
If a remote port is removed and remains removed for devloss_tmo, if an RSCN is subsequently received indicating the presence of the remte port, the driver does not login to and rediscovery the remote port. Currently, in order to for a port to be rediscovered post an RSCN, the node state must be NPR to reflect not logged in. When devloss expires, the node state is marked UNUSED. When an RSCN occurs, the nodes referenced by the RSCN will have a NPR_2B_DISC flag set, but the re-login will only be attempted if the node is in NPR_NODE state. Thus the node is skipped over. Fix by recognizing the NPR_2B_DISC and UNUSED and transition the node back to NPR state to allow the re-login to take place. Signed-off-by:
-
James Smart authored
If an admin updates lpfc's devloss_tmo sysfs attribute, the kernel will oops. Coding of a loop allowed a new value (rport) to be set/checked for null followed by an older value (remoteport) checked for null to allow progress where the new value, even though null, will be referenced. Rework the logic to validate and prevent any reference to the null ptr. Signed-off-by:
-
James Smart authored
It's possible for the driver to initiate an FLOGI and before it completes, another link down/up transition occurs requiring a new FLOGI. Currently, nothing is done to abort/noop the older FLOGI request to the adapter, so if this transition occurs and the FLOGI completion is received after the link down/up transition, the driver may erroneously act on the older FLOGI. In most cases, the adapter properly terminates/fails the FLOGI, but there is a timing condition where the FLOGI may complete on the wire prior to the transition, but the response may not be seen/processed by the driver before the driver sees the link transition. Fix by having the link down handler in the driver run through any outstanding ELS's and change the completion handler of the ELS so that it will be no-op'd and released. Signed-off-by:
-
James Smart authored
When tearing down the adapter for a reset, online/offline, or driver unload, the queue free routine would hit a GPF oops. This only occurs on conditions where the number of hardware queues created is fewer than the number of cpus in the system. In this condition cpus share a hardware queue. And of course, it's the 2nd cpu that shares a hardware that attempted to free it a second time and hit the oops. Fix by reworking the cpu to hardware queue mapping such that: Assignment of hardware queues to cpus occur in two passes: first pass: is first time assignment of a hardware queue to a cpu. This will set the LPFC_CPU_FIRST_IRQ flag for the cpu. second pass: for cpus that did not get a hardware queue they will be assigned one from a primary cpu (one set in first pass). Deletion of hardware queues is driven by cpu itteration, and queues will only be deleted if the LPFC_CPU_FIRST_IRQ flag is set. Also contains a few small cleanup fixes and a little better logging. Signed-off-by:
-
James Smart authored
The adapter reset path (lpfc_sli_hba_down) is taking/releasing a lock with irq. But, the path is already under the hbalock which raised irq so it's unnecessary. Convert to simple lock/unlock. Signed-off-by:
-
James Smart authored
lpfc_nvme_register_port hit a null prev_ndlp pointer in a test with lots of target ports swapping addresses. The oldport value was stale, thus it's ndlp (prev_ndlp set to it) was used. Fix by moving oldrport pointer checks, and if used prev_ndlp pointer assignment, to be done while the lock is held. Signed-off-by:
-
James Smart authored
The driver is inadvertently trying to issue an INIT_VPI mailbox command on an SLI-3 driver. The command is specific to SLI-4. When the call is made to send the command, if on an SLI-3 adapter, an array pointer is NULL and the driver will oops. Fix by restricting the command to SLI-4 adapters only. Signed-off-by:
-
James Smart authored
If a target issues an ADISC to the port and the target is a NVME target, the driver is inadvertantly invalidating the login and marking the remote port as logged out. Communication with the target is lost. Revise the ADISC check so that FCP or NVME targets will be marked valid at the end of ADISC processing. Enhance logging to recognize condition better. Signed-off-by:
-
James Smart authored
Some remote ports may be slow in registering their GID_FT protocol information with the fabric. If the remote port is an initiator, it may send PLOGI to the port before the GID_FT logic is complete. Meaning, after accepting the PLOGI, when the driver may see no response to the GID_FT that is issued after the login to determine the protocols supported so that proper PRLI's may be transmit. If the driver has no fc4 information, it currently stops and the remote port is not discovered. Fix by issuing a LOGO when there is no GID_FT information. The LOGO completion handling will attempt to re-login if the nport_id is still present. Signed-off-by:
-
James Smart authored
In cases of remote-port-side cable pull/replug, there happens to be a target that upon replug will send the port a PLOGI, a PRLI, and a LOGO. When this sequence is received by the driver, the PLOGI accepted and a GFT_ID is issued to find the protocol support for the remote port. While the GFT_ID is outstanding, a LOGO is received. The driver logs the remote port out and unregisters the RPI and schedules a new PLOGI transmission. However, the GFT_ID was not terminated. When it completed, the driver attempted to transition the remote port to PRLI transmission, which cancels the PLOGI scheduling. The PRLI transmit attempt is rejected by the adapter as the remote port is not logged in. No retry is attempted as it's expected the logout is noted and the supposedly scheduled PLOGI should address the state. As there is no PLOGI, the remote port does not get re-discovered. Fix by aborting the outstanding GFT_ID if the related remote port is logged out. Ensure a PRLI transmit attempt only occurs if the remote port is logging in. This avoids the incorrect attempt while logged out. Signed-off-by:
-
James Smart authored
If the adapter is reset while there are outstanding ELS's, subsequent reinitialization of the adapter will fail as it has not recovered all of the io contexts relative to the ELS's. If an ELS timed out or otherwise failed and an the ELS was attempted to be aborted (which changes the ELS completion context), in causes where the driver generates completions for the outstanding IO as the adapter would not due to being reset, the driver released only the ELS context and failed to release the abort context. When the adapter went to reinit, as it had not received all of the contexts, it failed to reinit. Fix by having the ELS completion handler identify the driver-generated completion status and release the abort context. Signed-off-by:
-
James Smart authored
Unusually high IO latency can be observed with little IO in progress. The latency may remain high regardless of amount of IO and can only be cleared by forcing lpfc_fcp_imax values to non-zero and then back to zero. The driver's eq_delay mechanism that scales the interrupt coalescing based on io completion load failed to reduce or turn off coalescing when load decreased. Specifically, if no io completed on a cpu within an eq_delay polling window, the eq delay processing was skipped and no change was made to the coalescing values. This left the coalescing values set when they were no longer applicable. Fix by always clearing the percpu counters for each time period and always run the eq_delay calculations if an eq has a non-zero coalescing value. Signed-off-by:
-
James Smart authored
If a timer routine uses workqueues, it could fire before the workqueue is allocated. Fix by allocating the workqueue before the timer routines are setup Signed-off-by:
-
James Smart authored
After seeing some interoperability issues with ADISC, it was determined the ELS definitions in lpfc were using types that allowed the compiler to add pad to the structure, causing the structure to no longer be per spec. The offending structures are ADISC, FAN, and RNID. This patch implements the simple fix of eliminating the pad by forcing the compiler to pack the structure. Care was taken to ensure field accesses won't be by operations that would hit a bad field alignment. The better solution would be to convert to the uapi fc header definitions, but the number of changes required to do is rather intrusive so this course of action was deferred. Signed-off-by:
-
James Smart authored
When connected to a high number of remote ports, the driver is encountering PLOGI errors. The errors are due to adapter detected failures indicating illegal field values. Turns out the driver was prematurely clearing an RPI bitmask before waiting for an UNREG_RPI mailbox completion. This allowed the RPI to be reused before it was actually available. Fix by clearing RPI bitmask only after UNREG_RPI mailbox completion. Signed-off-by:
-
James Smart authored
scsi-mq operation inherently performs pre-allocation of resources for blk-mq request queues. Even though the kdump environment reduces the configuration to a single CPU, thus 1 hardware queue, which helps significantly, the resources are still rather large due to the per request allocations. blk-mq pre-allocations can be over 4KB per request. With adapter can_queue values in the 4k or 8k range, this can easily be 32MBs before any other driver memory is factored in. Driver SGL DMA buffer allocation can be up to 8KB per request as well adding an additional 64MB. Totals are well over 100MB for a single shost. Given kdump memory auto-sizing utilities don't accommodate this amount of memory well, it's very possible for kdump to fail due to lack of memory. Fix by having the driver recognize that it is booting within a kdump context and reduce the number of requests it will support to a more reasonable value. Signed-off-by:
-
Hariprasad Kelam authored
As dma_pool_destroy and mempool_destroy functions has NULL check. We may not need NULL check before calling them. Fix below warnings reported by coccicheck ./drivers/scsi/lpfc/lpfc_mem.c:252:2-18: WARNING: NULL check before some freeing functions is not needed. ./drivers/scsi/lpfc/lpfc_mem.c:255:2-18: WARNING: NULL check before some freeing functions is not needed. ./drivers/scsi/lpfc/lpfc_mem.c:258:2-18: WARNING: NULL check before some freeing functions is not needed. ./drivers/scsi/lpfc/lpfc_mem.c:261:2-18: WARNING: NULL check before some freeing functions is not needed. ./drivers/scsi/lpfc/lpfc_mem.c:265:2-18: WARNING: NULL check before some freeing functions is not needed. ./drivers/scsi/lpfc/lpfc_mem.c:269:2-17: WARNING: NULL check before some freeing functions is not needed. Signed-off-by:
Hariprasad Kelam <hariprasad.kelam@gmail.com> Reviewed-by:
James Smart <james.smart@broadcom.com> Signed-off-by:
-
