• Hagen Paul Pfeifer's avatar
    x86/asm: Always inline atomics · 3462bd2a
    Hagen Paul Pfeifer authored
    During some code analysis I realized that atomic_add(), atomic_sub()
    and friends are not necessarily inlined AND that each function
    is defined multiple times:
    
    	atomic_inc:          544 duplicates
    	atomic_dec:          215 duplicates
    	atomic_dec_and_test: 107 duplicates
    	atomic64_inc:         38 duplicates
    	[...]
    
    Each definition is exact equally, e.g.:
    
    	ffffffff813171b8 <atomic_add>:
    	55         push   %rbp
    	48 89 e5   mov    %rsp,%rbp
    	f0 01 3e   lock add %edi,(%rsi)
    	5d         pop    %rbp
    	c3         retq
    
    In turn each definition has one or more callsites (sure):
    
    	ffffffff81317c78: e8 3b f5 ff ff  callq  ffffffff813171b8 <atomic_add> [...]
    	ffffffff8131a062: e8 51 d1 ff ff  callq  ffffffff813171b8 <atomic_add> [...]
    	ffffffff8131a190: e8 23 d0 ff ff  callq  ffffffff813171b8 <atomic_add> [...]
    
    The other way around would be to remove the static linkage - but
    I prefer an enforced inlining here.
    
    	Before:
    	  text     data	  bss      dec       hex     filename
    	  81467393 19874720 20168704 121510817 73e1ba1 vmlinux.orig
    
    	After:
    	  text     data     bss      dec       hex     filename
    	  81461323 19874720 20168704 121504747 73e03eb vmlinux.inlined
    
    Yes, the inlining here makes the kernel even smaller! ;)
    
    Linus further observed:
    
    	"I have this memory of having seen that before - the size
    	 heuristics for gcc getting confused by inlining.
    	 [...]
    
    	 It might be a good idea to mark things that are basically just
    	 wrappers around a single (or a couple of) asm instruction to be
    	 always_inline."
    Signed-off-by: default avatarHagen Paul Pfeifer <hagen@jauu.net>
    Acked-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    Cc: Borislav Petkov <bp@alien8.de>
    Cc: H. Peter Anvin <hpa@zytor.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Link: http://lkml.kernel.org/r/1429565231-4609-1-git-send-email-hagen@jauu.netSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
    3462bd2a
atomic64_64.h 5.04 KB