• Rik van Riel's avatar
    sched, time: Switch VIRT_CPU_ACCOUNTING_GEN to jiffy granularity · ff9a9b4c
    Rik van Riel authored
    When profiling syscall overhead on nohz-full kernels,
    after removing __acct_update_integrals() from the profile,
    native_sched_clock() remains as the top CPU user. This can be
    reduced by moving VIRT_CPU_ACCOUNTING_GEN to jiffy granularity.
    
    This will reduce timing accuracy on nohz_full CPUs to jiffy
    based sampling, just like on normal CPUs. It results in
    totally removing native_sched_clock from the profile, and
    significantly speeding up the syscall entry and exit path,
    as well as irq entry and exit, and KVM guest entry & exit.
    
    Additionally, only call the more expensive functions (and
    advance the seqlock) when jiffies actually changed.
    
    This code relies on another CPU advancing jiffies when the
    system is busy. On a nohz_full system, this is done by a
    housekeeping CPU.
    
    A microbenchmark calling an invalid syscall number 10 million
    times in a row speeds up an additional 30% over the numbers
    with just the previous patches, for a total speedup of about
    40% over 4.4 and 4.5-rc1.
    
    Run times for the microbenchmark:
    
     4.4				3.8 seconds
     4.5-rc1			3.7 seconds
     4.5-rc1 + first patch		3.3 seconds
     4.5-rc1 + first 3 patches	3.1 seconds
     4.5-rc1 + all patches		2.3 seconds
    
    A non-NOHZ_FULL cpu (not the housekeeping CPU):
    
     all kernels			1.86 seconds
    Signed-off-by: default avatarRik van Riel <riel@redhat.com>
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Reviewed-by: default avatarThomas Gleixner <tglx@linutronix.de>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Mike Galbraith <efault@gmx.de>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: clark@redhat.com
    Cc: eric.dumazet@gmail.com
    Cc: fweisbec@gmail.com
    Cc: luto@amacapital.net
    Link: http://lkml.kernel.org/r/1455152907-18495-5-git-send-email-riel@redhat.comSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
    ff9a9b4c
cputime.c 23.1 KB