e1000: add dynamic itr modes

Add a new dynamic itr algorithm, with 2 modes, and make it the default
operation mode. This greatly reduces latency and increases small packet
performance, at the "cost" of some CPU utilization. Bulk traffic
throughput is unaffected.

The driver can limit the amount of interrupts per second that the
adapter will generate for incoming packets. It does this by writing a
value to the adapter that is based on the maximum amount of interrupts
that the adapter will generate per second.

Setting InterruptThrottleRate to a value greater or equal to 100 will
program the adapter to send out a maximum of that many interrupts per
second, even if more packets have come in. This reduces interrupt
load on the system and can lower CPU utilization under heavy load,
but will increase latency as packets are not processed as quickly.

The default behaviour of the driver previously assumed a static
InterruptThrottleRate value of 8000, providing a good fallback value
for all traffic types,but lacking in small packet performance and
latency. The hardware can handle many more small packets per second
however, and for this reason an adaptive interrupt moderation algorithm
was implemented.

Since 7.3.x, the driver has two adaptive modes (setting 1 or 3) in
which it dynamically adjusts the InterruptThrottleRate value based on
the traffic that it receives. After determining the type of incoming
traffic in the last timeframe, it will adjust the InterruptThrottleRate
to an appropriate value for that traffic.

The algorithm classifies the incoming traffic every interval into
classes.  Once the class is determined, the InterruptThrottleRate
value is adjusted to suit that traffic type the best. There are
three classes defined: "Bulk traffic", for large amounts of packets
of normal size; "Low latency", for small amounts of traffic and/or
a significant percentage of small packets; and "Lowest latency",
for almost completely small packets or minimal traffic.

In dynamic conservative mode, the InterruptThrottleRate value is
set to 4000 for traffic that falls in class "Bulk traffic". If
traffic falls in the "Low latency" or "Lowest latency" class, the
InterruptThrottleRate is increased stepwise to 20000. This default
mode is suitable for most applications.

For situations where low latency is vital such as cluster or
grid computing, the algorithm can reduce latency even more when
InterruptThrottleRate is set to mode 1. In this mode, which operates
the same as mode 3, the InterruptThrottleRate will be increased
stepwise to 70000 for traffic in class "Lowest latency".

Setting InterruptThrottleRate to 0 turns off any interrupt moderation
and may improve small packet latency, but is generally not suitable
for bulk throughput traffic.
Signed-off-by: Jesse Brandeburg <jesse.brandeburg@intel.com>
Cc: Rick Jones <rick.jones2@hp.com>
Signed-off-by: Auke Kok <auke-jan.h.kok@intel.com>

drivers/net/e1000/e1000.h

@@ -257,6 +257,17 @@ struct e1000_adapter {
 	spinlock_t tx_queue_lock;
 #endif
 	atomic_t irq_sem;
+	unsigned int detect_link;
+	unsigned int total_tx_bytes;
+	unsigned int total_tx_packets;
+	unsigned int total_rx_bytes;
+	unsigned int total_rx_packets;
+	/* Interrupt Throttle Rate */
+	uint32_t itr;
+	uint32_t itr_setting;
+	uint16_t tx_itr;
+	uint16_t rx_itr;
+
 	struct work_struct reset_task;
 	uint8_t fc_autoneg;
 
@@ -314,8 +325,6 @@ struct e1000_adapter {
 	uint64_t gorcl_old;
 	uint16_t rx_ps_bsize0;
 
-	/* Interrupt Throttle Rate */
-	uint32_t itr;
 
 	/* OS defined structs */
 	struct net_device *netdev;

drivers/net/e1000/e1000_param.c

@@ -139,7 +139,7 @@ E1000_PARAM(XsumRX, "Disable or enable Receive Checksum offload");
  * Valid Range: 0-65535
  */
 E1000_PARAM(TxIntDelay, "Transmit Interrupt Delay");
-#define DEFAULT_TIDV                  64
+#define DEFAULT_TIDV                   8
 #define MAX_TXDELAY               0xFFFF
 #define MIN_TXDELAY                    0
 
@@ -148,7 +148,7 @@ E1000_PARAM(TxIntDelay, "Transmit Interrupt Delay");
  * Valid Range: 0-65535
  */
 E1000_PARAM(TxAbsIntDelay, "Transmit Absolute Interrupt Delay");
-#define DEFAULT_TADV                  64
+#define DEFAULT_TADV                  32
 #define MAX_TXABSDELAY            0xFFFF
 #define MIN_TXABSDELAY                 0
 
@@ -167,16 +167,16 @@ E1000_PARAM(RxIntDelay, "Receive Interrupt Delay");
  * Valid Range: 0-65535
  */
 E1000_PARAM(RxAbsIntDelay, "Receive Absolute Interrupt Delay");
-#define DEFAULT_RADV                 128
+#define DEFAULT_RADV                   8
 #define MAX_RXABSDELAY            0xFFFF
 #define MIN_RXABSDELAY                 0
 
 /* Interrupt Throttle Rate (interrupts/sec)
  *
- * Valid Range: 100-100000 (0=off, 1=dynamic)
+ * Valid Range: 100-100000 (0=off, 1=dynamic, 3=dynamic conservative)
  */
 E1000_PARAM(InterruptThrottleRate, "Interrupt Throttling Rate");
-#define DEFAULT_ITR                 8000
+#define DEFAULT_ITR                    3
 #define MAX_ITR                   100000
 #define MIN_ITR                      100
 
@@ -473,14 +473,26 @@ e1000_check_options(struct e1000_adapter *adapter)
 			case 1:
 				DPRINTK(PROBE, INFO, "%s set to dynamic mode\n",
 					opt.name);
+				adapter->itr_setting = adapter->itr;
+				adapter->itr = 20000;
+				break;
+			case 3:
+				DPRINTK(PROBE, INFO,
+				        "%s set to dynamic conservative mode\n",
+					opt.name);
+				adapter->itr_setting = adapter->itr;
+				adapter->itr = 20000;
 				break;
 			default:
 				e1000_validate_option(&adapter->itr, &opt,
 				        adapter);
+				/* save the setting, because the dynamic bits change itr */
+				adapter->itr_setting = adapter->itr;
 				break;
 			}
 		} else {
-			adapter->itr = opt.def;
+			adapter->itr_setting = opt.def;
+			adapter->itr = 20000;
 		}
 	}
 	{ /* Smart Power Down */