- 21 Jul, 2020 40 commits
-
-
Parav Pandit authored
Constify devlink instance pointer while checking if reload operation is supported or not. This helps to review the scope of checks done in reload. Signed-off-by:
Parav Pandit <parav@mellanox.com> Reviewed-by:
Jiri Pirko <jiri@mellanox.com> Signed-off-by:
David S. Miller <davem@davemloft.net>
-
Parav Pandit authored
Add comment to describe the purpose of devlink instance lock. Signed-off-by:
-
Parav Pandit authored
Reload operation is enabled or not is already checked by devlink_reload(). Hence, remove the duplicate check from devlink_nl_cmd_reload(). Signed-off-by:
-
Parav Pandit authored
There is no need to hold a device global lock when initializing devlink device fields of a devlink instance which is not yet part of the devices list. Signed-off-by:
-
Heiner Kallweit authored
For most chip versions this has been added already. Allow also for RTL8125A to enable ASPM. Signed-off-by:
Heiner Kallweit <hkallweit1@gmail.com> Signed-off-by:
-
David S. Miller authored
Arthur Kiyanovski says: ==================== ENA driver new features V4 changes: ----------- Add smp_rmb() to "net: ena: avoid unnecessary rearming of interrupt vector when busy-polling" to adhere to the linux kernel memory model, and update the commit message accordingly. V3 changes: ----------- 1. Add "net: ena: enable support of rss hash key and function changes" patch again, with more explanations why it should be in net-next in commit message. 2. Add synchronization considerations to "net: ena: avoid unnecessary rearming of interrupt vector when busy-polling" V2 changes: ----------- 1. Update commit messages of 2 patches to be more verbose. 2. Remove "net: ena: enable support of rss hash key and function changes" patch. Will be resubmitted net. V1 cover letter: ---------------- This patchset contains performance improvements, support for new devices and functionality: 1. Support for upcoming ENA devices 2. Avoid unnecessary IRQ unmasking in busy poll to reduce interrupt rate 3. Enabling device support for RSS function and key manipulation 4. Support for NIC-based traffic mirroring (SPAN port) 5. Additional PCI device ID 6. Cosmetic changes ==================== Acked-by:
Jakub Kicinski <kuba@kernel.org> Signed-off-by:
-
Arthur Kiyanovski authored
New devices add a new hardware acceleration engine, which adds some restrictions to the driver. Metadata descriptor must be present for each packet and the maximum burst size between two doorbells is now limited to a number advertised by the device. This patch adds: 1. A handshake protocol between the driver and the device, so the device will enable the accelerated queues only when both sides support it. 2. The driver support for the new acceleration engine: 2.1. Send metadata descriptor for each Tx packet. 2.2. Limit the number of packets sent between doorbells.(*) (*) A previous driver implementation of this feature was comitted in commit 05d62ca2 ("net: ena: add handling of llq max tx burst size") however the design of the interface between the driver and device changed since then. This change is reflected in this commit. Signed-off-by:
Netanel Belgazal <netanel@amazon.com> Signed-off-by:
Arthur Kiyanovski <akiyano@amazon.com> Signed-off-by:
-
Arthur Kiyanovski authored
When the ENA device resets to recover from some error state, all LLQ configuration values are reset to their defaults, because LLQ is initialized only once during ena_probe(). Changes in this commit: 1. Move the LLQ configuration process into ena_init_device() which is called from both ena_probe() and ena_restore_device(). This way, LLQ setup configurations that are different from the default values will survive resets. 2. Extract the LLQ bar mapping to ena_map_llq_bar(), and call once in the lifetime of the driver from ena_probe(), since there is no need to unmap and map the LLQ bar again every reset. 3. Map the LLQ bar if it exists, regardless if initialization of LLQ placement policy (ENA_ADMIN_PLACEMENT_POLICY_DEV) succeeded or not. Initialization might fail the first time, falling back to the ENA_ADMIN_PLACEMENT_POLICY_HOST placement policy, but later succeed after device reset, in which case the LLQ bar needs to be mapped already. Signed-off-by:
Sameeh Jubran <sameehj@amazon.com> Signed-off-by:
-
Arthur Kiyanovski authored
Add the rss_configurable_function_key bit to driver_supported_feature. This bit tells the device that the driver in question supports the retrieving and updating of RSS function and hash key, and therefore the device should allow RSS function and key manipulation. This commit turns on device support for hash key and RSS function management. Without this commit this feature is turned off at the device and appears to the user as unsupported. This commit concludes the following series of already merged commits: commit 0af3c4e2 ("net: ena: changes to RSS hash key allocation") commit c1bd17e5 ("net: ena: change default RSS hash function to Toeplitz") commit f66c2ea3 ("net: ena: allow setting the hash function without changing the key") commit e9a1de37 ("net: ena: fix error returning in ena_com_get_hash_function()") commit 80f8443f ("net: ena: avoid unnecessary admin command when RSS function set fails") commit 6a4f7dc8 ("net: ena: rss: do not allocate key when not supported") commit 0d1c3de7 ("net: ena: fix incorrect default RSS key") The above commits represent the last part of the implementation of this feature, and with them merged the feature can be enabled in the device. Signed-off-by:
-
Arthur Kiyanovski authored
Add support for traffic mirroring, where the hardware reads the buffer from the instance memory directly. Traffic Mirroring needs access to the rx buffers in the instance. To have this access, this patch: 1. Changes the code to map and unmap the rx buffers bidirectionally. 2. Enables the relevant bit in driver_supported_features to indicate to the FW that this driver supports traffic mirroring. Rx completion is not generated until mirroring is done to avoid the situation where the driver changes the buffer before it is mirrored. Signed-off-by:
-
Arthur Kiyanovski authored
The size of the admin statistics in ena_com_stats_admin is changed from 32bit to 64bit so to align with the sizes of the other statistics in the driver (i.e. rx_stats, tx_stats and ena_stats_dev). This is done as part of an effort to create a unified API to read statistics. Signed-off-by:
Shay Agroskin <shayagr@amazon.com> Signed-off-by:
-
Arthur Kiyanovski authored
gcc 4.8 reports a warning when initializing with = {0}. Dropping the "0" from the braces fixes the issue. This fix is not ANSI compatible but is allowed by gcc. Signed-off-by: -
Arthur Kiyanovski authored
Add a reserved PCI device ID to the driver's table Used for internal testing purposes. Signed-off-by:
-
Arthur Kiyanovski authored
For an overview of the race created by this patch goto synchronization label. In napi busy-poll mode, the kernel invokes the napi handler of the device repeatedly to poll the NIC's receive queues. This process repeats until a timeout, specific for each connection, is up. By polling packets in busy-poll mode the user may gain lower latency and higher throughput (since the kernel no longer waits for interrupts to poll the queues) in expense of CPU usage. Upon completing a napi routine, the driver checks whether the routine was called by an interrupt handler. If so, the driver re-enables interrupts for the device. This is needed since an interrupt routine invocation disables future invocations until explicitly re-enabled. The driver avoids re-enabling the interrupts if they were not disabled in the first place (e.g. if driver in busy mode). Originally, the driver checked whether interrupt re-enabling is needed by reading the 'ena_napi->unmask_interrupt' variable. This atomic variable was set upon interrupt and cleared after re-enabling it. In the 4.10 Linux version, the 'napi_complete_done' call was changed so that it returns 'false' when device should not re-enable interrupts, and 'true' otherwise. The change includes reading the "NAPIF_STATE_IN_BUSY_POLL" flag to check if the napi call is in busy-poll mode, and if so, return 'false'. The driver was changed to re-enable interrupts according to this routine's return value. The Linux community rejected the use of the 'ena_napi->unmaunmask_interrupt' variable to determine whether unmasking is needed, and urged to use napi_napi_complete_done() return value solely. See https://lore.kernel.org/patchwork/patch/741149/ for more details As explained, a busy-poll session exists for a specified timeout value, after which it exits the busy-poll mode and re-enters it later. This leads to many invocations of the napi handler where napi_complete_done() false indicates that interrupts should be re-enabled. This creates a bug in which the interrupts are re-enabled unnecessarily. To reproduce this bug: 1) echo 50 | sudo tee /proc/sys/net/core/busy_poll 2) echo 50 | sudo tee /proc/sys/net/core/busy_read 3) Add counters that check whether 'ena_unmask_interrupt(tx_ring, rx_ring);' is called without disabling the interrupts in the first place (i.e. with calling the interrupt routine ena_intr_msix_io()) Steps 1+2 enable busy-poll as the default mode for new connections. The busy poll routine rearms the interrupts after every session by design, and so we need to add an extra check that the interrupts were masked in the first place. synchronization: This patch introduces a race between the interrupt handler ena_intr_msix_io() and the napi routine ena_io_poll(). Some macros and instruction were added to prevent this race from leaving the interrupts masked. The following specifies the different race scenarios in this patch: 1) interrupt handler and napi routine run sequentially i) interrupt handler is called, sets 'interrupts_masked' flag and successfully schedules the napi handler via softirq. In this scenario the napi routine might not see the flag change for several reasons: a) The flag is stored in a register by the compiler. For this case the WRITE_ONCE macro which prevents this. b) The compiler might reorder the instruction. For this the smp_wmb() instruction was used which implies a compiler memory barrier. c) On archs with weak consistency model (like ARM64) the napi routine might be scheduled and start running before the flag STORE instruction is committed to cache/memory. To ensure this doesn't happen, the smp_wmb() instruction was added. It ensures that the flag set instruction is committed before scheduling napi. ii) compiler reorders the flag's value check in the 'if' with the flag set in the napi routine. This scenario is prevented by smp_rmb() call after the flag check. 2) interrupt handler and napi routine run in parallel (can happen when busy poll routine invokes the napi handler) i) interrupt handler sets the flag in one core, while the napi routine reads it in another core. This scenario also is divided into two cases: a) napi_complete_done() doesn't finish running, in which case napi_sched() would just set NAPIF_STATE_MISSED and the napi routine would reschedule itself without changing the flag's value. b) napi_complete_done() finishes running. In this case the napi routine might override the flag's value. This doesn't present any rise since it later unmasks the interrupt vector. Signed-off-by:
-
Yuval Basson authored
- Free ILT lines used for XRC-SRQ's contexts. - Free XRCD bitmap Fixes: b8204ad8 ("qed: changes to ILT to support XRC") Fixes: 7bfb399e ("qed: Add XRC to RoCE") Signed-off-by:
Michal Kalderon <mkalderon@marvell.com> Signed-off-by:
Yuval Basson <ybason@marvell.com> Signed-off-by:
-
David S. Miller authored
Russell King says: ==================== Phylink PCS updates This series updates the rudimentary phylink PCS support with the results of the last four months of development of that. Phylink PCS support was initially added back at the end of March, when it became clear that the current approach of treating everything at the MAC end as being part of the MAC was inadequate. However, this rudimentary implementation was fine initially for mvneta and similar, but in practice had a fair number of issues, particularly when ethtool interfaces were used to change various link properties. It became apparent that relying on the phylink_config structure for the PCS was also bad when it became clear that the same PCS was used in DSA drivers as well as in NXPs other offerings, and there was a desire to re-use that code. It also became apparent that splitting the "configuration" step on an interface mode configuration between the MAC and PCS using just mac_config() and pcs_config() methods was not sufficient for some setups, as the MAC needed to be "taken down" prior to making changes, and once all settings were complete, the MAC could only then be resumed. This series addresses these points, progressing PCS support, and has been developed with mvneta and DPAA2 setups, with work on both those drivers to prove this approach. It has been rigorously tested with mvneta, as that provides the most flexibility for testing the various code paths. To solve the phylink_config reuse problem, we introduce a struct phylink_pcs, which contains the minimal information necessary, and it is intended that this is embedded in the PCS private data structure. To solve the interface mode configuration problem, we introduce two new MAC methods, mac_prepare() and mac_finish() which wrap the entire interface mode configuration only. This has the additional benefit of relieving MAC drivers from working out whether an interface change has occurred, and whether they need to do some major work. I have not yet updated all the interface documentation for these changes yet, that work remains, but this patch set is provided in the hope that those working on PCS support in NXP will find this useful. Since there is a lot of change here, this is the reason why I strongly advise that everyone has converted to the mac_link_up() way of configuring the link parameters when the link comes up, rather than the old way of using mac_config() - especially as splitting the PCS changes how and when phylink calls mac_config(). Although no change for existing users is intended, that is something I no longer am able to test. Changes since RFC: - fix bisect build failure - add patch to use config.an_enabled - rename phylink_config_interface to phylink_major_reconfig - add expanded documentation for phylink_set_pcs() ==================== Signed-off-by: -
Russell King authored
Add an interface to configure the advertisement for a clause 22 PCS PHY, and set the AN enable flag in the BMCR appropriately. Reviewed-by:
Ioana Ciornei <ioana.ciornei@nxp.com> Signed-off-by:
Russell King <rmk+kernel@armlinux.org.uk> Signed-off-by:
-
Russell King authored
Add a way for MAC PCS to have private data while keeping independence from struct phylink_config, which is used for the MAC itself. We need this independence as we will have stand-alone code for PCS that is independent of the MAC. Introduce struct phylink_pcs, which is designed to be embedded in a driver private data structure. This structure does not include a mdio_device as there are PCS implementations such as the Marvell DSA and network drivers where this is not necessary. Reviewed-by:
-
Russell King authored
With PCS support, how we implement interface reconfiguration (or other major reconfiguration) is not up to the job; we end up reconfiguring the PCS for an interface change while the link could potentially be up. In order to solve this, add two additional MAC methods for major configuration, one to prepare for the change, and one to finish the change. This allows mvneta and mvpp2 to shutdown what they require prior to the MAC and PCS configuration calls, and then restart as appropriate. This impacts ksettings_set(), which now needs to identify whether the change is a minor tweak to the advertisement masks or whether the interface mode has changed, and call the appropriate function for that update. Signed-off-by:
-
Russell King authored
Re-code the pause in-band advertisement update in light of the addition of PCS support, so that we perform the minimum required; only the PCS configuration function needs to be called in this case, followed by the request to trigger a restart of negotiation if the programmed advertisement changed. We need to change the pcs_config() signature to pass whether resolved pause should be passed to the MAC for setups such as mvneta and mvpp2 where doing so overrides the MAC manual flow controls. Signed-off-by:
-
Russell King authored
For fixed links, we only allow the current settings, so this should be a matter of merely rejecting an attempt to change the settings. If the settings agree, then there is nothing more we need to do. Reviewed-by:
-
Russell King authored
Rather than recomputing whether AN is enabled, use config.an_enabled. Suggested-by:
-
Russell King authored
When we have a PHY attached, an ethtool ksettings_set() call only really needs to call through to the phylib equivalent; phylib will call back to us when the link changes so we can update our state. Therefore, we can bypass most of our ksettings_set() call for this case. Reviewed-by:
-
Russell King authored
Simplify the ksettings_set() implementation to look more like phylib's implementation; use a switch() for validating the autoneg setting, and use the linkmode_modify() helper to set the autoneg bit in the advertisement mask. Reviewed-by:
-
Russell King authored
Avoid calling mac_config() when using split PCS, and the interface remains the same. Reviewed-by:
Florian Fainelli <f.fainelli@gmail.com> Signed-off-by:
-
Russell King authored
The only PHYs that are used with phylink which change their interface are the BCM84881 and MV88X3310 family, both of which only change their interface modes on link-up events. This will break when drivers are converted to split-PCS. Fix this. Reviewed-by:
-
Russell King authored
The only PHYs that are used with phylink which change their interface are the BCM84881 and MV88X3310 family, both of which only change their interface modes on link-up events. However, rather than relying upon this behaviour by the PHY, we should give a stronger guarantee when resolving that the link will be down whenever we change the interface mode. This patch implements that stronger guarantee for resolve. Reviewed-by:
-
Russell King authored
Use a boolean to indicate whether mac_config() should be called during a resolution. This allows resolution to have a single location where mac_config() will be called, which will allow us to make decisions about how and what we do. Reviewed-by:
-
Russell King authored
Rejig the link state tracking, so that we can use the current state in a future patch. Reviewed-by:
-
Russell King authored
Comparing the ethtool output from phylink and non-phylink fixed-link setups shows that we have some differences: - The "auto-negotiation" fields are different; phylink reports these as "No", non-phylink reports these as "Yes" for the supported and advertising masks. - The link partner advertisement is set to the link speed with non- phylink, but phylink leaves this unset, causing all link partner fields to be omitted. The phylink ethtool output also disagrees with the software emulated PHY dump via the MII registers. Update the phylink fixed-link parsing code so that we better reflect the behaviour of the non-phylink code that this facility replaces, and bring the ethtool interface more into line with the report from via the MII interface. Reviewed-by:
-
David S. Miller authored
Claudiu Manoil says: ==================== enetc: Add adaptive interrupt coalescing Apart from some related cleanup patches, this set introduces in a straightforward way the support needed to enable and configure interrupt coalescing for ENETC. Patch 5 introduces the support needed for configuring the interrupt coalescing parameters and for switching between moderated (int. coalescing) and per-packet interrupt modes. When interrupt coalescing is enabled the Rx/Tx time thresholds are configurable, packet thresholds are fixed. To make this work reliably, patch 5 uses the traffic pause procedure introduced in patch 2. Patch 6 adds DIM (Dynamic Interrupt Moderation) to implement adaptive coalescing based on time thresholds, for the Rx 'channel'. On the Tx side a default optimal value is used instead, optimized for TCP traffic over 1G and 2.5G links. This default 'optimal' value can be overridden anytime via 'ethtool -C tx-usecs'. netperf -t TCP_MAERTS measurements show a significant CPU load reduction correlated w/ reduced interrupt rates. For the measurement results refer to the comments in patch 6. v2: Replaced Tx DIM with predefined optimal value, giving better results. This was also suggested by Jakub (cc). Switched order of patches 4 and 5, for better grouping. v3: minor cleanup/improvements ==================== Signed-off-by: -
Claudiu Manoil authored
Use the generic dynamic interrupt moderation (dim) framework to implement adaptive interrupt coalescing on Rx. With the per-packet interrupt scheme, a high interrupt rate has been noted for moderate traffic flows leading to high CPU utilization. The 'dim' scheme implemented by the current patch addresses this issue improving CPU utilization while using minimal coalescing time thresholds in order to preserve a good latency. On the Tx side use an optimal time threshold value by default. This value has been optimized for Tx TCP streams at a rate of around 85kpps on a 1G link, at which rate half of the Tx ring size (128) gets filled in 1500 usecs. Scaling this down to 2.5G links yields the current value of 600 usecs, which is conservative and gives good enough results for 1G links too (see next). Below are some measurement results for before and after this patch (and related dependencies) basically, for a 2 ARM Cortex-A72 @1.3Ghz CPUs system (32 KB L1 data cache), using 60secs log netperf TCP stream tests @ 1Gbit link (maximum throughput): 1) 1 Rx TCP flow, both Rx and Tx processed by the same NAPI thread on the same CPU: CPU utilization int rate (ints/sec) Before: 50%-60% (over 50%) 92k After: 13%-22% 3.5k-12k Comment: Major CPU utilization improvement for a single flow Rx TCP flow (i.e. netperf -t TCP_MAERTS) on a single CPU. Usually settles under 16% for longer tests. 2) 4 Rx TCP flows + 4 Tx TCP flows (+ pings to check the latency): Total CPU utilization Total int rate (ints/sec) Before: ~80% (spikes to 90%) ~100k After: 60% (more steady) ~4k Comment: Important improvement for this load test, while the ping test outcome does not show any notable difference compared to before. Signed-off-by:
Claudiu Manoil <claudiu.manoil@nxp.com> Signed-off-by:
-
Claudiu Manoil authored
Enable programming of the interrupt coalescing registers and allow manual configuration of the coalescing time thresholds via ethtool. Packet thresholds have been fixed to predetermined values as there's no point in making them run-time configurable, also anticipating the dynamic interrupt moderation (DIM) algorithm which uses fixed packet thresholds as well. If the interface is up when the operation mode of traffic interrupt events is changed by the user (i.e. switching from default per-packet interrupts to coalesced interrupts), the traffic needs to be paused in the process. This patch also prepares the ground for introducing DIM on Rx. Signed-off-by:
-
Claudiu Manoil authored
'struct enetc_bdr' is already '____cacheline_aligned_in_smp'. Signed-off-by:
-
Claudiu Manoil authored
Interrupt coalescing registers naming in the current revision of the Ref Man (RM) is ICR, deprecating the ICIR name used in earlier (draft) versions of the RM. Signed-off-by:
-
Claudiu Manoil authored
A reliable traffic pause (and reconfiguration) procedure is needed to be able to safely make h/w configuration changes during run-time, like changing the mode in which the interrupts are operating (i.e. with or without coalescing), as opposed to making on-the-fly register updates that may be subject to h/w or s/w concurrency issues. To this end, the code responsible of the run-time device configurations that basically starts resp. stops the traffic flow through the device has been extracted from the the enetc_open/_close procedures, to the separate standalone enetc_start/_stop procedures. Traffic stop should be as graceful as possible, it lets the executing napi threads to to finish while the interrupts stay disabled. But since the napi thread will try to re-enable interrupts by clearing the device's unmask register, the enable_irq/ disable_irq API has been used to avoid this potential concurrency issue and make the traffic pause procedure more reliable. Signed-off-by:
-
Claudiu Manoil authored
It's time to differentiate between Rx and Tx ring sizes. Not only Tx rings are processed differently than Rx rings, but their default number also differs - i.e. up to 8 Tx rings per device (8 traffic classes) vs. 2 Rx rings (one per CPU). So let's set Tx rings sizes to half the size of the Rx rings for now, to be conservative. The default ring sizes were decreased as well (to the next lower power of 2), to reduce the memory footprint, buffering etc., since the measurements I've made so far show that the rings are very unlikely to get full. This change also anticipates the introduction of the dynamic interrupt moderation (dim) algorithm which operates on maximum packet thresholds of 256 packets for Rx and 128 packets for Tx. Signed-off-by:
-
Jisheng Zhang authored
Use devm_gpiod_get_array() to simplify the error handling and exit code path. Signed-off-by:
Jisheng Zhang <Jisheng.Zhang@synaptics.com> Signed-off-by:
-
Vladimir Oltean authored
Now that DSA supports MTU configuration, undo the effects of commit 8b1efc0f ("net: remove MTU limits on a few ether_setup callers") and let DSA interfaces use the default min_mtu and max_mtu specified by ether_setup(). This is more important for min_mtu: since DSA is Ethernet, the minimum MTU is the same as of any other Ethernet interface, and definitely not zero. For the max_mtu, we have a callback through which drivers can override that, if they want to. Signed-off-by:
Vladimir Oltean <olteanv@gmail.com> Reviewed-by:
-
Jonathan McDowell authored
This switch has a single max frame size configuration register, so we track the requested MTU for each port and apply the largest. v2: - Address review feedback from Vladimir Oltean Signed-off-by:
Jonathan McDowell <noodles@earth.li> Acked-by:
-
