• Eric Dumazet's avatar
    irq: use per_cpu kstat_irqs · 6c9ae009
    Eric Dumazet authored
    Use modern per_cpu API to increment {soft|hard}irq counters, and use
    per_cpu allocation for (struct irq_desc)->kstats_irq instead of an array.
    
    This gives better SMP/NUMA locality and saves few instructions per irq.
    
    With small nr_cpuids values (8 for example), kstats_irq was a small array
    (less than L1_CACHE_BYTES), potentially source of false sharing.
    
    In the !CONFIG_SPARSE_IRQ case, remove the huge, NUMA/cache unfriendly
    kstat_irqs_all[NR_IRQS][NR_CPUS] array.
    
    Note: we still populate kstats_irq for all possible irqs in
    early_irq_init().  We probably could use on-demand allocations.  (Code
    included in alloc_descs()).  Problem is not all IRQS are used with a prior
    alloc_descs() call.
    
    kstat_irqs_this_cpu() is not used anymore, remove it.
    Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
    Reviewed-by: default avatarChristoph Lameter <cl@linux.com>
    Cc: Ingo Molnar <mingo@elte.hu>
    Cc: Andi Kleen <andi@firstfloor.org>
    Cc: Tejun Heo <tj@kernel.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    6c9ae009
irqdesc.c 9.58 KB