- 30 Apr, 2020 15 commits
-
-
Peter Zijlstra authored
Currently objtool cannot understand retpolines, and thus cannot generate ORC unwind information for them. This means that we cannot unwind from the middle of a retpoline. The recent ANNOTATE_INTRA_FUNCTION_CALL and UNWIND_HINT_RET_OFFSET support in objtool enables it to understand the basic retpoline construct. A further problem is that the ORC unwind information is alternative invariant; IOW. every alternative should have the same ORC, retpolines obviously violate this. This means we need to out-of-line them. Since all GCC generated code already uses out-of-line retpolines, this should not affect performance much, if anything. This will enable objtool to generate valid ORC data for the out-of-line copies, which means we can correctly and reliably unwind through a retpoline. Signed-off-by:
Peter Zijlstra (Intel) <peterz@infradead.org> Acked-by:
Josh Poimboeuf <jpoimboe@redhat.com> Link: https://lkml.kernel.org/r/20200428191700.210835357@infradead.org
-
Peter Zijlstra authored
In order to change the {JMP,CALL}_NOSPEC macros to call out-of-line versions of the retpoline magic, we need to remove the '%' from the argument, such that we can paste it onto symbol names. Signed-off-by: -
Peter Zijlstra authored
Because of how KSYM works, we need one declaration per line. Seeing how we're going to be doubling the amount of retpoline symbols, simplify the machinery in order to avoid having to copy/paste even more. Signed-off-by:
-
Peter Zijlstra authored
Change FILL_RETURN_BUFFER so that objtool groks it and can generate correct ORC unwind information. - Since ORC is alternative invariant; that is, all alternatives should have the same ORC entries, the __FILL_RETURN_BUFFER body can not be part of an alternative. Therefore, move it out of the alternative and keep the alternative as a sort of jump_label around it. - Use the ANNOTATE_INTRA_FUNCTION_CALL annotation to white-list these 'funny' call instructions to nowhere. - Use UNWIND_HINT_EMPTY to 'fill' the speculation traps, otherwise objtool will consider them unreachable. - Move the RSP adjustment into the loop, such that the loop has a deterministic stack layout. Signed-off-by:
Alexandre Chartre <alexandre.chartre@oracle.com> Acked-by:
-
Alexandre Chartre authored
Change objtool to support intra-function calls. On x86, an intra-function call is represented in objtool as a push onto the stack (of the return address), and a jump to the destination address. That way the stack information is correctly updated and the call flow is still accurate. Signed-off-by:
Miroslav Benes <mbenes@suse.cz> Acked-by:
-
Miroslav Benes authored
Quoting Julien: "And the other suggestion is my other email was that you don't even need to add INSN_EXCEPTION_RETURN. You can keep IRET as INSN_CONTEXT_SWITCH by default and x86 decoder lookups the symbol conaining an iret. If it's a function symbol, it can just set the type to INSN_OTHER so that it caries on to the next instruction after having handled the stack_op." Suggested-by:
Julien Thierry <jthierry@redhat.com> Signed-off-by:
-
Peter Zijlstra authored
With the unconditional use of handle_insn_ops(), INSN_STACK has lost its purpose. Remove it. Suggested-by:
-
Peter Zijlstra authored
Now that every instruction has a list of stack_ops; we can trivially distinquish those instructions that do not have stack_ops, their list is empty. This means we can now call handle_insn_ops() unconditionally. Suggested-by:
-
Peter Zijlstra authored
Wrap each stack_op in a macro that allocates and adds it to the list. This simplifies trying to figure out what to do with the pre-allocated stack_op at the end. Signed-off-by:
-
Alexandre Chartre authored
UNWIND_HINT_RET_OFFSET will adjust a modified stack. However if a callee-saved register was pushed on the stack then the stack frame will still appear modified. So stop checking registers when UNWIND_HINT_RET_OFFSET is used. Signed-off-by:
-
Alexandre Chartre authored
Fix is_fentry_call() so that it works if a call has no destination set (call_dest). This needs to be done in order to support intra- function calls. Signed-off-by:
-
Peter Zijlstra authored
As reported by objtool: lib/ubsan.o: warning: objtool: .altinstr_replacement+0x0: alternative modifies stack lib/ubsan.o: warning: objtool: .altinstr_replacement+0x7: alternative modifies stack the smap_{save,restore}() alternatives violate (the newly enforced) rule on stack invariance. That is, due to there only being a single ORC table it must be valid to any alternative. These alternatives violate this with the direct result that unwinds will not be correct when it hits between the PUSH and POP instructions. Rewrite the functions to only have a conditional jump. Signed-off-by: -
Peter Zijlstra authored
Jann reported that (for instance) entry_64.o:general_protection has very odd ORC data: 0000000000000f40 <general_protection>: #######sp:sp+8 bp:(und) type:iret end:0 f40: 90 nop #######sp:(und) bp:(und) type:call end:0 f41: 90 nop f42: 90 nop #######sp:sp+8 bp:(und) type:iret end:0 f43: e8 a8 01 00 00 callq 10f0 <error_entry> #######sp:sp+0 bp:(und) type:regs end:0 f48: f6 84 24 88 00 00 00 testb $0x3,0x88(%rsp) f4f: 03 f50: 74 00 je f52 <general_protection+0x12> f52: 48 89 e7 mov %rsp,%rdi f55: 48 8b 74 24 78 mov 0x78(%rsp),%rsi f5a: 48 c7 44 24 78 ff ff movq $0xffffffffffffffff,0x78(%rsp) f61: ff ff f63: e8 00 00 00 00 callq f68 <general_protection+0x28> f68: e9 73 02 00 00 jmpq 11e0 <error_exit> #######sp:(und) bp:(und) type:call end:0 f6d: 0f 1f 00 nopl (%rax) Note the entry at 0xf41. Josh found this was the result of commit: 764eef4b ("objtool: Rewrite alt->skip_orig") Due to the early return in validate_branch() we no longer set insn->cfi of the original instruction stream (the NOPs at 0xf41 and 0xf42) and we'll end up with the above weirdness. In other discussions we realized alternatives should be ORC invariant; that is, due to there being only a single ORC table, it must be valid for all alternatives. The easiest way to ensure this is to not allow any stack modifications in alternatives. When we enforce this latter observation, we get the property that the whole alternative must have the same CFI, which we can employ to fix the former report. Fixes: 764eef4b ("objtool: Rewrite alt->skip_orig") Reported-by:Jann Horn <jannh@google.com> Signed-off-by:
-
Alexandre Chartre authored
Assign a unique identifier to every alternative instruction group in order to be able to tell which instructions belong to what alternative. [peterz: extracted from a larger patch] Signed-off-by:
-
Julien Thierry authored
While jumping from outside an alternative region to the middle of an alternative region is very likely wrong, jumping from an alternative region into the same region is valid. It is a common pattern on arm64. The first pattern is unlikely to happen in practice and checking only for this adds a lot of complexity. Just remove the current check. Suggested-by:
-
- 23 Apr, 2020 3 commits
-
-
Ingo Molnar authored
Mostly straightforward constification, except that WARN_FUNC() needs a writable pointer while we have read-only pointers, so deflect this to WARN(). Acked-by:
Ingo Molnar <mingo@kernel.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Sami Tolvanen <samitolvanen@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: https://lore.kernel.org/r/20200422103205.61900-4-mingo@kernel.org
-
Ingo Molnar authored
'struct elf *' handling is an open/close paradigm, make sure the naming matches that: elf_open_read() elf_write() elf_close() Acked-by:
-
Ingo Molnar authored
In preparation to parallelize certain parts of objtool, map out which uses of various data structures are read-only vs. read-write. As a first step constify 'struct elf' pointer passing, most of the secondary uses of it in find_symbol_*() methods are read-only. Also, while at it, better group the 'struct elf' handling methods in elf.h. Acked-by:
-
- 22 Apr, 2020 22 commits
-
-
Thomas Gleixner authored
Consider all of .entry.text as noinstr. This gets us coverage across the PTI boundary. While we could add everything .noinstr.text into .entry.text that would bloat the amount of code in the user mapping. Signed-off-by:
Thomas Gleixner <tglx@linutronix.de> Signed-off-by:
-
Peter Zijlstra authored
Make sure to also check STT_NOTYPE symbols for noinstr violations. Signed-off-by:
-
Peter Zijlstra authored
In preparation of further changes, once again break out the loop body. No functional changes intended. Signed-off-by:
-
Peter Zijlstra authored
validate_functions() iterates all sections their symbols; this is pointless to do for !text sections as they won't have instructions anyway. Signed-off-by:
-
Peter Zijlstra authored
Now that objtool is capable of processing vmlinux.o and actually has something useful to do there, (conditionally) add it to the final link pass. This will increase build time by a few seconds. Signed-off-by:
-
Peter Zijlstra authored
In preparation for find_insn_containing(), change insn_hash to use sec_offset_hash(). This actually reduces runtime; probably because mixing in the section index reduces the collisions due to text sections all starting their instructions at offset 0. Runtime on vmlinux.o from 3.1 to 2.5 seconds. Signed-off-by:
-
Peter Zijlstra authored
When doing kbuild tests to see if the objtool changes affected those I found that there was a measurable regression: pre post real 1m13.594 1m16.488s user 34m58.246s 35m23.947s sys 4m0.393s 4m27.312s Perf showed that for small files the increased hash-table sizes were a measurable difference. Since we already have -l "vmlinux" to distinguish between the modes, make it also use a smaller portion of the hash-tables. This flips it into a small win: real 1m14.143s user 34m49.292s sys 3m44.746s Signed-off-by: -
Peter Zijlstra authored
Validate that any call out of .noinstr.text is in between instr_begin() and instr_end() annotations. This annotation is useful to ensure correct behaviour wrt tracing sensitive code like entry/exit and idle code. When we run code in a sensitive context we want a guarantee no unknown code is ran. Since this validation relies on knowing the section of call destination symbols, we must run it on vmlinux.o instead of on individual object files. Add two options: -d/--duplicate "duplicate validation for vmlinux" -l/--vmlinux "vmlinux.o validation" Where the latter auto-detects when objname ends with "vmlinux.o" and the former will force all validations, also those already done on !vmlinux object files. Signed-off-by:
-
Peter Zijlstra authored
Objtool keeps per instruction CFI state in struct insn_state and will save/restore this where required. However, insn_state has grown some !CFI state, and this must not be saved/restored (that would loose/destroy state). Fix this by moving the CFI specific parts of insn_state into struct cfi_state. Signed-off-by:
-
Peter Zijlstra authored
There's going to be a new struct cfi_state, rename this one to make place. Signed-off-by:
-
Peter Zijlstra authored
The SAVE/RESTORE hints are now unused; remove them. Signed-off-by:
-
Peter Zijlstra authored
'Optimize' ftrace_regs_caller. Instead of comparing against an immediate, the more natural way to test for zero on x86 is: 'test %r,%r'. 48 83 f8 00 cmp $0x0,%rax 74 49 je 226 <ftrace_regs_call+0xa3> 48 85 c0 test %rax,%rax 74 49 je 225 <ftrace_regs_call+0xa2> Signed-off-by:
-
Peter Zijlstra authored
There's a convenient macro for 'SS+8' called FRAME_SIZE. Use it to clarify things. (entry/calling.h calls this SIZEOF_PTREGS but we're using asm/ptrace-abi.h) Signed-off-by:
-
Peter Zijlstra authored
The ftrace_regs_caller() trampoline does something 'funny' when there is a direct-caller present. In that case it stuffs the 'direct-caller' address on the return stack and then exits the function. This then results in 'returning' to the direct-caller with the exact registers we came in with -- an indirect tail-call without using a register. This however (rightfully) confuses objtool because the function shares a few instruction in order to have a single exit path, but the stack layout is different for them, depending through which path we came there. This is currently cludged by forcing the stack state to the non-direct case, but this generates actively wrong (ORC) unwind information for the direct case, leading to potential broken unwinds. Fix this issue by fully separating the exit paths. This results in having to poke a second RET into the trampoline copy, see ftrace_regs_caller_ret. This brings us to a second objtool problem, in order for it to perceive the 'jmp ftrace_epilogue' as a function exit, it needs to be recognised as a tail call. In order to make that happen, ftrace_epilogue needs to be the start of an STT_FUNC, so re-arrange code to make this so. Finally, a third issue is that objtool requires functions to exit with the same stack layout they started with, which is obviously violated in the direct case, employ the new HINT_RET_OFFSET to tell objtool this is an expected exception. Together, this results in generating correct ORC unwind information for the ftrace_regs_caller() function and it's trampoline copies. Signed-off-by:
-
Peter Zijlstra authored
Normally objtool ensures a function keeps the stack layout invariant. But there is a useful exception, it is possible to stuff the return stack in order to 'inject' a 'call': push $fun ret In this case the invariant mentioned above is violated. Add an objtool HINT to annotate this and allow a function exit with a modified stack frame. Signed-off-by:
-
Peter Zijlstra authored
Teach objtool a little more about IRET so that we can avoid using the SAVE/RESTORE annotation. In particular, make the weird corner case in insn->restore go away. The purpose of that corner case is to deal with the fact that UNWIND_HINT_RESTORE lands on the instruction after IRET, but that instruction can end up being outside the basic block, consider: if (cond) sync_core() foo(); Then the hint will land on foo(), and we'll encounter the restore hint without ever having seen the save hint. By teaching objtool about the arch specific exception frame size, and assuming that any IRET in an STT_FUNC symbol is an exception frame sized POP, we can remove the use of save/restore hints for this code. Signed-off-by:
-
Julien Thierry authored
Instruction sets can include more or less complex operations which might not fit the currently defined set of stack_ops. Combining more than one stack_op provides more flexibility to describe the behaviour of an instruction. This also reduces the need to define new stack_ops specific to a single instruction set. Allow instruction decoders to generate multiple stack_op per instruction. Signed-off-by:
-
Muchun Song authored
If the prefix of section name is not '.rodata', the following function call can never return 0. strcmp(sec->name, C_JUMP_TABLE_SECTION) So the name comparison is pointless, just remove it. Signed-off-by:Muchun Song <songmuchun@bytedance.com> Signed-off-by:
-
Nick Desaulniers authored
Compiling with Clang and CONFIG_KASAN=y was exposing a few warnings: call to memset() with UACCESS enabled Document how to fix these for future travelers. Link: https://github.com/ClangBuiltLinux/linux/issues/876Suggested-by:
Kamalesh Babulal <kamalesh@linux.vnet.ibm.com> Suggested-by:
Matt Helsley <mhelsley@vmware.com> Suggested-by:
Randy Dunlap <rdunlap@infradead.org> Signed-off-by:
Nick Desaulniers <ndesaulniers@google.com> Signed-off-by:
-
Julien Thierry authored
Some CFI definitions used by generic objtool code have no reason to vary from one architecture to another. Keep those definitions in generic code and move the arch-specific ones to a new arch-specific header. Signed-off-by:
-
Raphael Gault authored
The jump and call destination relocation offsets are x86-specific. Abstract them by calling arch-specific implementations. [ jthierry: Remove superfluous comment; replace other addend offsets with arch_dest_rela_offset() ] Signed-off-by:Raphael Gault <raphael.gault@arm.com> Signed-off-by:
-
Julien Thierry authored
The initial register state is set up by arch specific code. Use the value the arch code has set when restoring registers from the stack. Suggested-by:
-
