- 08 Feb, 2021 40 commits
-
-
Nicholas Piggin authored
This moves the 64s/hash context tracking from hash_page_mm() to __do_hash_fault(), so it's no longer called by OCXL / SPU accelerators, which was certainly the wrong thing to be doing, because those callers are not low level interrupt handlers, so should have entered a kernel context tracking already. Then remain in kernel context for the duration of the fault, rather than enter/exit for the hash fault then enter/exit for the page fault, which is pointless. Even still, calling exception_enter/exit in __do_hash_fault seems questionable because that's touching per-cpu variables, tracing, etc., which might have been interrupted by this hash fault or themselves cause hash faults. But maybe I miss something because hash_page_mm very deliberately calls trace_hash_fault too, for example. So for now go with it, it's no worse than before, in this regard. Signed-off-by:
Nicholas Piggin <npiggin@gmail.com> Signed-off-by:
Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20210130130852.2952424-32-npiggin@gmail.com
-
Nicholas Piggin authored
Add context tracking to the system call handler explicitly, and remove _TIF_NOHZ. This improves system call performance when nohz_full is enabled. On a POWER9, gettid scv system call cost on a nohz_full CPU improves from 1129 cycles to 1004 cycles and on a housekeeping CPU from 550 cycles to 430 cycles. Signed-off-by:
-
Nicholas Piggin authored
Simple helper for synchronous interrupt handlers (i.e., process-context) to enable interrupts if it was taken in an interrupts-enabled context. Signed-off-by:
-
Nicholas Piggin authored
Signed-off-by:
-
Nicholas Piggin authored
Move the program check handling into a function called by both, rather than have the emulation assist handler call the program check handler. This allows each of these handlers to be implemented with "interrupt wrappers" in a later change. Signed-off-by:
-
Nicholas Piggin authored
These will be used by subsequent patches. Signed-off-by:
-
Nicholas Piggin authored
Add wrapper functions (derived from x86 macros) for interrupt handler functions. This allows interrupt entry code to be written in C. Signed-off-by:
-
Nicholas Piggin authored
If an unrecoverable system reset hits in process context, the system does not have to panic. Similar to machine check, call nmi_exit() before die(). Signed-off-by:
-
Nicholas Piggin authored
A machine check that is handled must still check MSR[RI] for recoverability of the interrupted context. Without this patch it's possible for a handled machine check to return to a context where it has clobbered live registers. Signed-off-by:
-
Nicholas Piggin authored
As explained by commit daf00ae7 ("powerpc/traps: restore recoverability of machine_check interrupts"), die() can't be called from within nmi_enter to nicely kill a process context that was interrupted. nmi_exit must be called first. This adds a function die_mce which takes care of this for machine check handlers. Signed-off-by:
-
Nicholas Piggin authored
These are declared in ras.h and defined in ras.c so remove them from pervasive.h Signed-off-by:
-
Nicholas Piggin authored
This is currently the same as unknown_exception, but it will diverge after interrupt wrappers are added and code moved out of asm into the wrappers (e.g., async handlers will check FINISH_NAP). Signed-off-by:
-
Nicholas Piggin authored
Interrupt handler prototypes are going to be rearranged in a future patch, so tidy this out of the way first. Signed-off-by:
-
Nicholas Piggin authored
This is required in order to allow more significant differences between NMI type interrupt handlers and regular asynchronous handlers. Signed-off-by:
-
Nicholas Piggin authored
These NMIs could fire any time including inside kprobe code, so exclude them from kprobes. Signed-off-by:
-
Nicholas Piggin authored
This makes a small improvement to the description of the SLB interrupt environment. Move the memory access restrictions into one paragraph, and the interrupt restrictions into the next rather than mix them. Signed-off-by:
-
Nicholas Piggin authored
SLB faults no longer call do_page_fault, this was removed somewhere between 2.6.0 and 2.6.12. Signed-off-by:
-
Nicholas Piggin authored
This is required for subsequent interrupt wrapper implementation. Signed-off-by:
-
Nicholas Piggin authored
This simplifies code, and it is also useful when introducing interrupt handler wrappers when introducing wrapper functionality that doesn't cope with asm entry code calling into more than one handler function. 32-bit and 64e still have some such cases, which limits some ways they can use interrupt wrappers. Signed-off-by:
-
Nicholas Piggin authored
This keeps the context tracking over the entire interrupt handler which helps later with moving context tracking into interrupt wrappers. Signed-off-by:
-
Nicholas Piggin authored
This function acts like an interrupt handler so it needs to follow the standard interrupt handler function signature which will be introduced in a future change. Signed-off-by:
-
Nicholas Piggin authored
Similar to the previous patch this makes interrupt handler function types more regular so they can be wrapped with the next patch. Signed-off-by:
-
Nicholas Piggin authored
Now that handlers get all registers from pt_regs, r4 and r5 are no longer live here and may be clobbered. Signed-off-by:
-
Nicholas Piggin authored
Like other interrupt handler conversions, switch to getting registers from the pt_regs argument. Signed-off-by:
-
Nicholas Piggin authored
Similar to the previous patch this makes interrupt handler function types more regular so they can be wrapped with the next patch. Signed-off-by:
-
Nicholas Piggin authored
Like other interrupt handler conversions, switch to getting registers from the pt_regs argument. Signed-off-by:
-
Nicholas Piggin authored
Make mm fault handlers all just take the pt_regs * argument and load DAR/DSISR from that. Make those that return a value return long. This is done to make the function signatures match other handlers, which will help with a future patch to add wrappers. Explicit arguments could be added for performance but that would require more wrapper macro variants. Signed-off-by:
-
Nicholas Piggin authored
The fault handling still has some complex logic particularly around hash table handling, in asm. Implement most of this in C. Signed-off-by:
-
Nicholas Piggin authored
Similar to the 32/s change, move the test and call to the do_break handler to the DSI. Suggested-by:
Christophe Leroy <christophe.leroy@csgroup.eu> Signed-off-by:
-
Christophe Leroy authored
handle_page_fault() has some code dedicated to book3s/32 to call do_break() when the DSI is a DABR match. On other platforms, do_break() is handled separately. Do the same for book3s/32, do it earlier in the process of DSI. This change also avoid doing the test on ISI. Signed-off-by:
-
Nicholas Piggin authored
Interrupts that occur in kernel mode expect that context tracking is set to kernel. Enabling local irqs before context tracking switches from guest to host means interrupts can come in and trigger warnings about wrong context, and possibly worse. Signed-off-by:
-
Nicholas Piggin authored
When replaying pending soft-masked interrupts when an interrupt returns to an irqs-enabled context, there is a special case required if this was an asynchronous interrupt to avoid unbounded interrupt recursion. This case was not tested for in the case the asynchronous interrupt hit in user context, because a subsequent nested interrupt would by definition hit in kernel mode, which then exits via the kernel path which does test this case. There is no reason to allow this for such interrupts. While recursion is bounded at the next level, it's simpler and uses less stack to apply the replay logic consistently. This also expands the comment which was really pretty poor and didn't explain the problem (I can say that because I wrote it). Signed-off-by:
-
Oliver O'Halloran authored
Signed-off-by:
Oliver O'Halloran <oohall@gmail.com> Signed-off-by:
-
Oliver O'Halloran authored
Signed-off-by:
-
Oliver O'Halloran authored
Signed-off-by:
-
Oliver O'Halloran authored
Signed-off-by:
-
Oliver O'Halloran authored
Signed-off-by:
-
Oliver O'Halloran authored
Signed-off-by:
-
Oliver O'Halloran authored
Signed-off-by:
-
Oliver O'Halloran authored
Signed-off-by:
-
