scsi: megaraid_sas: Set affinity for high IOPS reply queues

High iops queues are mapped to non-managed IRQs. Set affinity of
non-managed irqs to local numa node.  Low latency queues are mapped to
managed IRQs.

Driver reserves some reply queues for high IOPS queues (through
pci_alloc_irq_vectors_affinity and .pre_vectors interface). The rest of
queues are for low latency.

Based on IO workload, driver will decide which group of reply queues
(either high IOPS queues or low latency queues) to be used.
High IOPS queues will be mapped to local numa node of controller and
low latency queues will be mapped to CPUs across numa nodes. In general,
high IOPS and low latency queues should fit into 128 reply queues
which is the max number of reply queues supported by Aero adapters.
Signed-off-by: Kashyap Desai <kashyap.desai@broadcom.com>
Signed-off-by: Chandrakanth Patil <chandrakanth.patil@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>

drivers/scsi/megaraid/megaraid_sas_base.c

@@ -5672,6 +5672,26 @@ int megasas_get_device_list(struct megasas_instance *instance)
 	return SUCCESS;
 }
 
+/**
+ * megasas_set_high_iops_queue_affinity_hint -	Set affinity hint for high IOPS queues
+ * @instance:					Adapter soft state
+ * return:					void
+ */
+static inline void
+megasas_set_high_iops_queue_affinity_hint(struct megasas_instance *instance)
+{
+	int i;
+	int local_numa_node;
+
+	if (instance->balanced_mode) {
+		local_numa_node = dev_to_node(&instance->pdev->dev);
+
+		for (i = 0; i < instance->low_latency_index_start; i++)
+			irq_set_affinity_hint(pci_irq_vector(instance->pdev, i),
+				cpumask_of_node(local_numa_node));
+	}
+}
+
 static int
 __megasas_alloc_irq_vectors(struct megasas_instance *instance)
 {
@@ -5729,6 +5749,8 @@ megasas_alloc_irq_vectors(struct megasas_instance *instance)
 	else
 		instance->msix_vectors = 0;
 
+	if (instance->smp_affinity_enable)
+		megasas_set_high_iops_queue_affinity_hint(instance);
 }
 
 /**