Commit ff967900 authored by Nicholas Piggin's avatar Nicholas Piggin Committed by Michael Ellerman

powerpc/64: Fix latency tracing for lazy irq replay

When returning from an exception to a soft-enabled context, pending
IRQs are replayed but IRQ tracing is not reset, so a number of them
can get chained together into the same IRQ-disabled trace.

Fix this by having __check_irq_replay re-set IRQ trace. This is
conceptually where we respond to the next interrupt, so it fits the
semantics of the IRQ tracer.
Signed-off-by: default avatarNicholas Piggin <npiggin@gmail.com>
Signed-off-by: default avatarMichael Ellerman <mpe@ellerman.id.au>
parent 6de6638b
...@@ -143,6 +143,13 @@ notrace unsigned int __check_irq_replay(void) ...@@ -143,6 +143,13 @@ notrace unsigned int __check_irq_replay(void)
*/ */
unsigned char happened = local_paca->irq_happened; unsigned char happened = local_paca->irq_happened;
/*
* We are responding to the next interrupt, so interrupt-off
* latencies should be reset here.
*/
trace_hardirqs_on();
trace_hardirqs_off();
if (happened & PACA_IRQ_HARD_DIS) { if (happened & PACA_IRQ_HARD_DIS) {
/* Clear bit 0 which we wouldn't clear otherwise */ /* Clear bit 0 which we wouldn't clear otherwise */
local_paca->irq_happened &= ~PACA_IRQ_HARD_DIS; local_paca->irq_happened &= ~PACA_IRQ_HARD_DIS;
...@@ -270,6 +277,7 @@ notrace void arch_local_irq_restore(unsigned long en) ...@@ -270,6 +277,7 @@ notrace void arch_local_irq_restore(unsigned long en)
#endif /* CONFIG_TRACE_IRQFLAGS */ #endif /* CONFIG_TRACE_IRQFLAGS */
set_soft_enabled(0); set_soft_enabled(0);
trace_hardirqs_off();
/* /*
* Check if anything needs to be re-emitted. We haven't * Check if anything needs to be re-emitted. We haven't
...@@ -279,6 +287,7 @@ notrace void arch_local_irq_restore(unsigned long en) ...@@ -279,6 +287,7 @@ notrace void arch_local_irq_restore(unsigned long en)
replay = __check_irq_replay(); replay = __check_irq_replay();
/* We can soft-enable now */ /* We can soft-enable now */
trace_hardirqs_on();
set_soft_enabled(1); set_soft_enabled(1);
/* /*
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment