- 08 Feb, 2021 40 commits
-
-
Nicholas Piggin authored
There is no need for this to be in asm, use the new intrrupt entry wrapper. Signed-off-by:
Nicholas Piggin <npiggin@gmail.com> Signed-off-by:
Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20210130130852.2952424-42-npiggin@gmail.com
-
Nicholas Piggin authored
Saving and restoring soft-mask state can now be done in C using the interrupt handler wrapper functions. Signed-off-by:
-
Nicholas Piggin authored
This moves the common NMI entry and exit code into the interrupt handler wrappers. This changes the behaviour of soft-NMI (watchdog) and HMI interrupts, and also MCE interrupts on 64e, by adding missing parts of the NMI entry to them. Signed-off-by:
-
Nicholas Piggin authored
The pseries real-mode machine check handler can enable the MMU, and return from the handler with the MMU still enabled. This works, but real-mode handler wrapper exit handlers want to rely on the MMU being in real-mode. So change the pseries handler to restore the MSR after it has finished virtual mode tasks. Signed-off-by:
-
Nicholas Piggin authored
There is no need for this to be in asm, use the new interrupt entry wrapper. Signed-off-by:
-
Nicholas Piggin authored
This will be used by interrupt entry as well. Signed-off-by:
-
Nicholas Piggin authored
There is no need for this to be in asm, use the new intrrupt entry wrapper. Signed-off-by:
-
Nicholas Piggin authored
The interrupt handler wrapper functions are not the ideal place to maintain context tracking because after they return, the low level exit code must then determine if there are interrupts to replay, or if the task should be preempted, etc. Those paths (e.g., schedule_user) include their own exception_enter/exit pairs to fix this up but it's a bit hacky (see schedule_user() comments). Ideally context tracking will go to user mode only when there are no more interrupts or context switches or other exit processing work to handle. 64e can not do this because it does not use the C interrupt exit code. Signed-off-by:
-
Nicholas Piggin authored
Move irq_enter/irq_exit into asynchronous interrupt handler wrappers. Signed-off-by:
-
Nicholas Piggin authored
Previously context tracking was not done for asynchronous interrupts, (those that run in interrupt context), and if those would cause a reschedule when they exit, then scheduling functions (schedule_user, preempt_schedule_irq) call exception_enter/exit to fix this up and exit user context. This is a hack we would like to get away from, so do context tracking for asynchronous interrupts too. Signed-off-by:
-
Nicholas Piggin authored
This moves exception_enter/exit calls to wrapper functions for synchronous interrupts. More interrupt handlers are covered by this than previously. Signed-off-by:
-
Nicholas Piggin authored
This moves the 64s/hash context tracking from hash_page_mm() to __do_hash_fault(), so it's no longer called by OCXL / SPU accelerators, which was certainly the wrong thing to be doing, because those callers are not low level interrupt handlers, so should have entered a kernel context tracking already. Then remain in kernel context for the duration of the fault, rather than enter/exit for the hash fault then enter/exit for the page fault, which is pointless. Even still, calling exception_enter/exit in __do_hash_fault seems questionable because that's touching per-cpu variables, tracing, etc., which might have been interrupted by this hash fault or themselves cause hash faults. But maybe I miss something because hash_page_mm very deliberately calls trace_hash_fault too, for example. So for now go with it, it's no worse than before, in this regard. Signed-off-by:
-
Nicholas Piggin authored
Add context tracking to the system call handler explicitly, and remove _TIF_NOHZ. This improves system call performance when nohz_full is enabled. On a POWER9, gettid scv system call cost on a nohz_full CPU improves from 1129 cycles to 1004 cycles and on a housekeeping CPU from 550 cycles to 430 cycles. Signed-off-by:
-
Nicholas Piggin authored
Simple helper for synchronous interrupt handlers (i.e., process-context) to enable interrupts if it was taken in an interrupts-enabled context. Signed-off-by:
-
Nicholas Piggin authored
Signed-off-by:
-
Nicholas Piggin authored
Move the program check handling into a function called by both, rather than have the emulation assist handler call the program check handler. This allows each of these handlers to be implemented with "interrupt wrappers" in a later change. Signed-off-by:
-
Nicholas Piggin authored
These will be used by subsequent patches. Signed-off-by:
-
Nicholas Piggin authored
Add wrapper functions (derived from x86 macros) for interrupt handler functions. This allows interrupt entry code to be written in C. Signed-off-by:
-
Nicholas Piggin authored
If an unrecoverable system reset hits in process context, the system does not have to panic. Similar to machine check, call nmi_exit() before die(). Signed-off-by:
-
Nicholas Piggin authored
A machine check that is handled must still check MSR[RI] for recoverability of the interrupted context. Without this patch it's possible for a handled machine check to return to a context where it has clobbered live registers. Signed-off-by:
-
Nicholas Piggin authored
As explained by commit daf00ae7 ("powerpc/traps: restore recoverability of machine_check interrupts"), die() can't be called from within nmi_enter to nicely kill a process context that was interrupted. nmi_exit must be called first. This adds a function die_mce which takes care of this for machine check handlers. Signed-off-by:
-
Nicholas Piggin authored
These are declared in ras.h and defined in ras.c so remove them from pervasive.h Signed-off-by:
-
Nicholas Piggin authored
This is currently the same as unknown_exception, but it will diverge after interrupt wrappers are added and code moved out of asm into the wrappers (e.g., async handlers will check FINISH_NAP). Signed-off-by:
-
Nicholas Piggin authored
Interrupt handler prototypes are going to be rearranged in a future patch, so tidy this out of the way first. Signed-off-by:
-
Nicholas Piggin authored
This is required in order to allow more significant differences between NMI type interrupt handlers and regular asynchronous handlers. Signed-off-by:
-
Nicholas Piggin authored
These NMIs could fire any time including inside kprobe code, so exclude them from kprobes. Signed-off-by:
-
Nicholas Piggin authored
This makes a small improvement to the description of the SLB interrupt environment. Move the memory access restrictions into one paragraph, and the interrupt restrictions into the next rather than mix them. Signed-off-by:
-
Nicholas Piggin authored
SLB faults no longer call do_page_fault, this was removed somewhere between 2.6.0 and 2.6.12. Signed-off-by:
-
Nicholas Piggin authored
This is required for subsequent interrupt wrapper implementation. Signed-off-by:
-
Nicholas Piggin authored
This simplifies code, and it is also useful when introducing interrupt handler wrappers when introducing wrapper functionality that doesn't cope with asm entry code calling into more than one handler function. 32-bit and 64e still have some such cases, which limits some ways they can use interrupt wrappers. Signed-off-by:
-
Nicholas Piggin authored
This keeps the context tracking over the entire interrupt handler which helps later with moving context tracking into interrupt wrappers. Signed-off-by:
-
Nicholas Piggin authored
This function acts like an interrupt handler so it needs to follow the standard interrupt handler function signature which will be introduced in a future change. Signed-off-by:
-
Nicholas Piggin authored
Similar to the previous patch this makes interrupt handler function types more regular so they can be wrapped with the next patch. Signed-off-by:
-
Nicholas Piggin authored
Now that handlers get all registers from pt_regs, r4 and r5 are no longer live here and may be clobbered. Signed-off-by:
-
Nicholas Piggin authored
Like other interrupt handler conversions, switch to getting registers from the pt_regs argument. Signed-off-by:
-
Nicholas Piggin authored
Similar to the previous patch this makes interrupt handler function types more regular so they can be wrapped with the next patch. Signed-off-by:
-
Nicholas Piggin authored
Like other interrupt handler conversions, switch to getting registers from the pt_regs argument. Signed-off-by:
-
Nicholas Piggin authored
Make mm fault handlers all just take the pt_regs * argument and load DAR/DSISR from that. Make those that return a value return long. This is done to make the function signatures match other handlers, which will help with a future patch to add wrappers. Explicit arguments could be added for performance but that would require more wrapper macro variants. Signed-off-by:
-
Nicholas Piggin authored
The fault handling still has some complex logic particularly around hash table handling, in asm. Implement most of this in C. Signed-off-by:
-
Nicholas Piggin authored
Similar to the 32/s change, move the test and call to the do_break handler to the DSI. Suggested-by:
Christophe Leroy <christophe.leroy@csgroup.eu> Signed-off-by:
-
