- 30 Jan, 2023 1 commit
-
-
Mark Brown authored
The KVM rseq test is failing to build in -next due to a commit merged from the tip tree which adds a wrapper for sys_getcpu() to the rseq kselftests, conflicting with the wrapper already included in the KVM selftest: rseq_test.c:48:13: error: conflicting types for 'sys_getcpu' 48 | static void sys_getcpu(unsigned *cpu) | ^~~~~~~~~~ In file included from rseq_test.c:23: ../rseq/rseq.c:82:12: note: previous definition of 'sys_getcpu' was here 82 | static int sys_getcpu(unsigned *cpu, unsigned *node) | ^~~~~~~~~~ Fix this by removing the local wrapper and moving the result check up to the caller. Fixes: 99babd04 ("selftests/rseq: Implement rseq numa node id field selftest") Signed-off-by:Mark Brown <broonie@kernel.org> Signed-off-by:
Peter Zijlstra (Intel) <peterz@infradead.org> Acked-by:
Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Link: https://lore.kernel.org/r/20230106-fix-kvm-rseq-build-v1-1-b704d9831d02@kernel.org
-
- 20 Jan, 2023 1 commit
-
-
Nicholas Piggin authored
If a task oopses with irqs disabled, this can cause various cascading problems in the oops path such as sleep-from-invalid warnings, and potentially worse. Since commit 0258b5fd ("coredump: Limit coredumps to a single thread group"), the unconditional irq enable in coredump_task_exit() will "fix" the irq state to be enabled early in do_exit(), so currently this may not be triggerable, but that is coincidental and fragile. Detect and fix the irqs_disabled() condition in the oops path before calling do_exit(), similarly to the way in_atomic() is handled. Reported-by:
Michael Ellerman <mpe@ellerman.id.au> Signed-off-by:
Nicholas Piggin <npiggin@gmail.com> Signed-off-by:
"Eric W. Biederman" <ebiederm@xmission.com> Link: https://lore.kernel.org/lkml/20221004094401.708299-1-npiggin@gmail.com/
-
- 18 Jan, 2023 2 commits
-
-
Peter Zijlstra authored
The recent cpuidle changes started triggering RCU splats on Juno development boards: | ============================= | WARNING: suspicious RCU usage | ----------------------------- | include/trace/events/ipi.h:19 suspicious rcu_dereference_check() usage! Fix cpuidle on ARM64: - ... by introducing a new 'is_rcu' flag to the cpuidle helpers & make ARM64 use it, as ARM64 wants to keep RCU active longer and wants to do the ct_cpuidle_enter()/exit() dance itself. - Also update the PSCI driver accordingly. - This also removes the last known RCU_NONIDLE() user as a bonus. Reported-by:
Mark Rutland <mark.rutland@arm.com> Signed-off-by:
Ingo Molnar <mingo@kernel.org> Tested-by:
Sudeep Holla <sudeep.holla@arm.com> Tested-by:
-
Arnd Bergmann authored
The added '.flags' value is sometimes ignored here because it gets overwritten by another initialization: drivers/cpuidle/cpuidle-mvebu-v7.c:24:33: error: initialized field overwritten [-Werror=override-init] 24 | #define MVEBU_V7_FLAG_DEEP_IDLE 0x10000 | ^~~~~~~ drivers/cpuidle/cpuidle-mvebu-v7.c:69:43: note: in expansion of macro 'MVEBU_V7_FLAG_DEEP_IDLE' ... Merge the two fields into one. Fixes: 4ce40e9d ("cpuidle, armada: Push RCU-idle into driver") Signed-off-by:Arnd Bergmann <arnd@arndb.de> Signed-off-by:
-
- 15 Jan, 2023 1 commit
-
-
Vincent Guittot authored
In presence of a lot of small weight tasks like sched_idle tasks, normal or high weight tasks can see their ideal runtime (sched_slice) to increase to hundreds ms whereas it normally stays below sysctl_sched_latency. 2 normal tasks running on a CPU will have a max sched_slice of 12ms (half of the sched_period). This means that they will make progress every sysctl_sched_latency period. If we now add 1000 idle tasks on the CPU, the sched_period becomes 3006 ms and the ideal runtime of the normal tasks becomes 609 ms. It will even become 1500ms if the idle tasks belongs to an idle cgroup. This means that the scheduler will look for picking another waiting task after 609ms running time (1500ms respectively). The idle tasks change significantly the way the 2 normal tasks interleave their running time slot whereas they should have a small impact. Such long sched_slice can delay significantly the release of resources as the tasks can wait hundreds of ms before the next running slot just because of idle tasks queued on the rq. Cap the ideal_runtime to sysctl_sched_latency to make sure that tasks will regularly make progress and will not be significantly impacted by idle/background tasks queued on the rq. Signed-off-by:
Vincent Guittot <vincent.guittot@linaro.org> Signed-off-by:
Dietmar Eggemann <dietmar.eggemann@arm.com> Link: https://lore.kernel.org/r/20230113133613.257342-1-vincent.guittot@linaro.org
-
- 13 Jan, 2023 35 commits
-
-
Peter Zijlstra authored
Low level noinstr context-tracking code is calling out to instrumented code on KASAN: vmlinux.o: warning: objtool: __ct_user_enter+0x72: call to __kasan_check_write() leaves .noinstr.text section vmlinux.o: warning: objtool: __ct_user_exit+0x47: call to __kasan_check_write() leaves .noinstr.text section Use even lower level atomic methods to avoid the instrumentation. Signed-off-by:
-
Peter Zijlstra authored
Add a few words on noinstr / __cpuidle usage. Signed-off-by:
-
Peter Zijlstra authored
For all cpuidle drivers that do not use CPUIDLE_FLAG_RCU_IDLE (iow, the simple ones) make sure all the functions are marked __cpuidle. ( due to lack of noinstr validation on these platforms it is entirely possible this isn't complete ) Signed-off-by:
-
Peter Zijlstra authored
For all cpuidle drivers that use CPUIDLE_FLAG_RCU_IDLE, ensure that all functions that call ct_cpuidle_enter() are marked __cpuidle. ( due to lack of noinstr validation on these platforms it is entirely possible this isn't complete ) Signed-off-by:
-
Peter Zijlstra authored
Tracing (kprobes included) and other compiler instrumentation relies on a normal kernel runtime. Therefore all functions that disable RCU should be noinstr, as should all functions that are called while RCU is disabled. Signed-off-by:
-
Peter Zijlstra authored
The PM notifiers should no longer be ran with RCU disabled (per the previous patches), as such this hack is no longer required either. Signed-off-by:
Tony Lindgren <tony@atomide.com> Tested-by:
Ulf Hansson <ulf.hansson@linaro.org> Acked-by:
Rafael J. Wysocki <rafael.j.wysocki@intel.com> Acked-by:
Frederic Weisbecker <frederic@kernel.org> Link: https://lore.kernel.org/r/20230112195542.151174682@infradead.org
-
Peter Zijlstra authored
Quite a few unnecessary instrumentation calls are generated via the no-op __this_cpu_preempt_check() call, if it gets uninlined by the compiler: vmlinux.o: warning: objtool: in_entry_stack+0x9: call to __this_cpu_preempt_check() leaves .noinstr.text section vmlinux.o: warning: objtool: default_do_nmi+0x10: call to __this_cpu_preempt_check() leaves .noinstr.text section vmlinux.o: warning: objtool: fpu_idle_fpregs+0x41: call to __this_cpu_preempt_check() leaves .noinstr.text section vmlinux.o: warning: objtool: kvm_read_and_reset_apf_flags+0x1: call to __this_cpu_preempt_check() leaves .noinstr.text section vmlinux.o: warning: objtool: lockdep_hardirqs_on+0xb0: call to __this_cpu_preempt_check() leaves .noinstr.text section vmlinux.o: warning: objtool: lockdep_hardirqs_off+0xae: call to __this_cpu_preempt_check() leaves .noinstr.text section vmlinux.o: warning: objtool: irqentry_nmi_enter+0x69: call to __this_cpu_preempt_check() leaves .noinstr.text section vmlinux.o: warning: objtool: irqentry_nmi_exit+0x32: call to __this_cpu_preempt_check() leaves .noinstr.text section vmlinux.o: warning: objtool: acpi_processor_ffh_cstate_enter+0x9: call to __this_cpu_preempt_check() leaves .noinstr.text section vmlinux.o: warning: objtool: acpi_idle_enter+0x43: call to __this_cpu_preempt_check() leaves .noinstr.text section vmlinux.o: warning: objtool: acpi_idle_enter_s2idle+0x45: call to __this_cpu_preempt_check() leaves .noinstr.text section Mark it __always_inline. Signed-off-by:
-
Peter Zijlstra authored
KASAN cannot just hijack the mem*() functions, it needs to emit __asan_mem*() variants if it wants instrumentation (other sanitizers already do this). vmlinux.o: warning: objtool: sync_regs+0x24: call to memcpy() leaves .noinstr.text section vmlinux.o: warning: objtool: vc_switch_off_ist+0xbe: call to memcpy() leaves .noinstr.text section vmlinux.o: warning: objtool: fixup_bad_iret+0x36: call to memset() leaves .noinstr.text section vmlinux.o: warning: objtool: __sev_get_ghcb+0xa0: call to memcpy() leaves .noinstr.text section vmlinux.o: warning: objtool: __sev_put_ghcb+0x35: call to memcpy() leaves .noinstr.text section Remove the weak aliases to ensure nobody hijacks these functions and add them to the noinstr section. Signed-off-by:
-
Peter Zijlstra authored
For testing purposes. Signed-off-by:
-
Peter Zijlstra authored
clang-14 allyesconfig gives: vmlinux.o: warning: objtool: emulator_cmpxchg_emulated+0x705: call to __ubsan_handle_load_invalid_value() with UACCESS enabled vmlinux.o: warning: objtool: paging64_update_accessed_dirty_bits+0x39e: call to __ubsan_handle_load_invalid_value() with UACCESS enabled vmlinux.o: warning: objtool: paging32_update_accessed_dirty_bits+0x390: call to __ubsan_handle_load_invalid_value() with UACCESS enabled vmlinux.o: warning: objtool: ept_update_accessed_dirty_bits+0x43f: call to __ubsan_handle_load_invalid_value() with UACCESS enabled Add the required eflags save/restore and whitelist the thing. Signed-off-by:
-
Peter Zijlstra authored
OMAP was the one and only user. Signed-off-by:
Stephen Boyd <sboyd@kernel.org> Link: https://lore.kernel.org/r/20230112195541.844982902@infradead.org
-
Peter Zijlstra authored
OMAP was the one and only user. Signed-off-by:
-
Peter Zijlstra authored
arch_cpu_idle() is a very simple idle interface and exposes only a single idle state and is expected to not require RCU and not do any tracing/instrumentation. As such, omap2_pm_idle() is not a valid implementation. Replace it with a simple (shallow) omap2_do_wfi() call. Omap2 doesn't have a cpuidle driver; but adding one would be the recourse to (re)gain the other idle states. Suggested-by:
-
Tony Lindgren authored
OMAP4 uses full SoC suspend modes as idle states, as such it needs the whole power-domain and clock-domain code from the idle path. All that code is not suitable to run with RCU disabled, as such push RCU-idle deeper still. Signed-off-by:
-
Peter Zijlstra authored
OMAP3 uses full SoC suspend modes as idle states, as such it needs the whole power-domain and clock-domain code from the idle path. All that code is not suitable to run with RCU disabled, as such push RCU-idle deeper still. Signed-off-by:
-
Peter Zijlstra authored
arch_cpu_idle() is a very simple idle interface and exposes only a single idle state and is expected to not require RCU and not do any tracing/instrumentation. As such, omap_sram_idle() is not a valid implementation. Replace it with the simple (shallow) omap3_do_wfi() call. Leaving the more complicated idle states for the cpuidle driver. Signed-off-by:
-
Peter Zijlstra authored
Robot reported that trace_hardirqs_{on,off}() tickle the forbidden _rcuidle() tracepoint through local_irq_{en,dis}able(). For 'sane' configs, these calls will only happen with RCU enabled and as such can use the regular tracepoint. This also means it's possible to trace them from NMI context again. Reported-by:kernel test robot <lkp@intel.com> Signed-off-by:
-
Peter Zijlstra authored
ARCH_WANTS_NO_INSTR (a superset of CONFIG_GENERIC_ENTRY) disallows any and all tracing when RCU isn't enabled. Signed-off-by:
-
Peter Zijlstra authored
Per commit 56e62a73 ("s390: convert to generic entry") the last and only callers of trace_hardirqs_{on,off}_caller() went away, clean up. Signed-off-by:
-
Peter Zijlstra authored
objtool found cases where ACPI methods called out into instrumentation code: vmlinux.o: warning: objtool: io_idle+0xc: call to __inb.isra.0() leaves .noinstr.text section vmlinux.o: warning: objtool: acpi_idle_enter+0xfe: call to num_online_cpus() leaves .noinstr.text section vmlinux.o: warning: objtool: acpi_idle_enter+0x115: call to acpi_idle_fallback_to_c1.isra.0() leaves .noinstr.text section Fix this by: marking the IO in/out, acpi_idle_fallback_to_c1() and num_online_cpus() methods as __always_inline. Signed-off-by:
-
Peter Zijlstra authored
objtool found that the mds_idle_clear_cpu_buffers() method got uninlined by the compiler where it called out into instrumentation: vmlinux.o: warning: objtool: mwait_idle+0x47: call to mds_idle_clear_cpu_buffers() leaves .noinstr.text section vmlinux.o: warning: objtool: acpi_processor_ffh_cstate_enter+0xa2: call to mds_idle_clear_cpu_buffers() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle+0x91: call to mds_idle_clear_cpu_buffers() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle_s2idle+0x8c: call to mds_idle_clear_cpu_buffers() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle_irq+0xaa: call to mds_idle_clear_cpu_buffers() leaves .noinstr.text section Solve this by marking mds_idle_clear_cpu_buffers() as __always_inline. Signed-off-by:
-
Peter Zijlstra authored
objtool found a few cases where this code called out into instrumented code: vmlinux.o: warning: objtool: acpi_idle_enter_s2idle+0xde: call to wbinvd() leaves .noinstr.text section vmlinux.o: warning: objtool: default_idle+0x4: call to arch_safe_halt() leaves .noinstr.text section vmlinux.o: warning: objtool: xen_safe_halt+0xa: call to HYPERVISOR_sched_op.constprop.0() leaves .noinstr.text section Solve this by: - marking arch_safe_halt(), wbinvd(), native_wbinvd() and HYPERVISOR_sched_op() as __always_inline(). - Explicitly uninlining xen_safe_halt() and pv_native_wbinvd() [they were already uninlined by the compiler on use as function pointers] and annotating them as 'noinstr'. - Annotating pv_native_safe_halt() as 'noinstr'. Signed-off-by:
Srivatsa S. Bhat (VMware) <srivatsa@csail.mit.edu> Reviewed-by:
Juergen Gross <jgross@suse.com> Acked-by:
-
Peter Zijlstra authored
objtool found a few cases where this code called out into instrumented code: vmlinux.o: warning: objtool: __halt+0x2c: call to hcall_func.constprop.0() leaves .noinstr.text section vmlinux.o: warning: objtool: __halt+0x3f: call to __tdx_hypercall() leaves .noinstr.text section vmlinux.o: warning: objtool: __tdx_hypercall+0x66: call to __tdx_hypercall_failed() leaves .noinstr.text section Fix it by: - moving TDX tdcall assembly methods into .noinstr.text (they are already noistr-clean) - marking __tdx_hypercall_failed() as 'noinstr' - annotating hcall_func() as __always_inline Signed-off-by:
-
Peter Zijlstra authored
objtool found a few cases where this code called out into instrumented code: vmlinux.o: warning: objtool: intel_idle_s2idle+0x6e: call to __monitor.constprop.0() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle_irq+0x8c: call to __monitor.constprop.0() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle+0x73: call to __monitor.constprop.0() leaves .noinstr.text section vmlinux.o: warning: objtool: mwait_idle+0x88: call to clflush() leaves .noinstr.text section Fix it by marking the affected methods as __always_inline. Signed-off-by:
-
Peter Zijlstra authored
objtool pointed out that various idle-TIF management methods have instrumentation: vmlinux.o: warning: objtool: mwait_idle+0x5: call to current_set_polling_and_test() leaves .noinstr.text section vmlinux.o: warning: objtool: acpi_processor_ffh_cstate_enter+0xc5: call to current_set_polling_and_test() leaves .noinstr.text section vmlinux.o: warning: objtool: cpu_idle_poll.isra.0+0x73: call to test_ti_thread_flag() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle+0xbc: call to current_set_polling_and_test() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle_irq+0xea: call to current_set_polling_and_test() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle_s2idle+0xb4: call to current_set_polling_and_test() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle+0xa6: call to current_clr_polling() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle_irq+0xbf: call to current_clr_polling() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle_s2idle+0xa1: call to current_clr_polling() leaves .noinstr.text section vmlinux.o: warning: objtool: mwait_idle+0xe: call to __current_set_polling() leaves .noinstr.text section vmlinux.o: warning: objtool: acpi_processor_ffh_cstate_enter+0xc5: call to __current_set_polling() leaves .noinstr.text section vmlinux.o: warning: objtool: cpu_idle_poll.isra.0+0x73: call to test_ti_thread_flag() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle+0xbc: call to __current_set_polling() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle_irq+0xea: call to __current_set_polling() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle_s2idle+0xb4: call to __current_set_polling() leaves .noinstr.text section vmlinux.o: warning: objtool: cpu_idle_poll.isra.0+0x73: call to test_ti_thread_flag() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle_s2idle+0x73: call to test_ti_thread_flag.constprop.0() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle_irq+0x91: call to test_ti_thread_flag.constprop.0() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle+0x78: call to test_ti_thread_flag.constprop.0() leaves .noinstr.text section vmlinux.o: warning: objtool: acpi_safe_halt+0xf: call to test_ti_thread_flag.constprop.0() leaves .noinstr.text section Remove the instrumentation, because these methods are used in low-level cpuidle code moving between states, that should not be instrumented. Signed-off-by:
-
Peter Zijlstra authored
No callers left that have already disabled RCU. Signed-off-by:
-
Peter Zijlstra authored
The problem, per commit fc98c3c8 ("printk: use rcuidle console tracepoint"), was printk usage from the cpuidle path where RCU was already disabled. Per the patches earlier in this series, this is no longer the case. Signed-off-by:
Sergey Senozhatsky <senozhatsky@chromium.org> Acked-by:
Petr Mladek <pmladek@suse.com> Acked-by:
-
Peter Zijlstra authored
Ever since commit d3afc7f1 ("arm64: Allow IPIs to be handled as normal interrupts") this function is called in regular IRQ context. Signed-off-by:
Marc Zyngier <maz@kernel.org> Acked-by:
-
Peter Zijlstra authored
None of these functions should ever be ran with RCU disabled anymore. Specifically, do_handle_IPI() is only called from handle_IPI() which explicitly does irq_enter()/irq_exit() which ensures RCU is watching. The problem with smp_cross_call() was, per commit description: 7c64cc05 ("arm: Use _rcuidle for smp_cross_call() tracepoints") ... that cpuidle_enter_state_coupled() already had RCU disabled, but that's long been fixed by commit: 1098582a ("sched,idle,rcu: Push rcu_idle deeper into the idle path") Signed-off-by:
-
Peter Zijlstra authored
Now that arch_cpu_idle() is expected to return with IRQs disabled, avoid the useless STI/CLI dance. Per the specs this is supposed to work, but nobody has yet relied up this behaviour so broken implementations are possible. Signed-off-by:
-
Peter Zijlstra authored
Current arch_cpu_idle() is called with IRQs disabled, but will return with IRQs enabled. However, the very first thing the generic code does after calling arch_cpu_idle() is raw_local_irq_disable(). This means that architectures that can idle with IRQs disabled end up doing a pointless 'enable-disable' dance. Therefore, push this IRQ disabling into the idle function, meaning that those architectures can avoid the pointless IRQ state flipping. Signed-off-by:
Gautham R. Shenoy <gautham.shenoy@amd.com> Acked-by: Mark Rutland <mark.rutland@arm.com> [arm64] Acked-by:
Guo Ren <guoren@kernel.org> Acked-by:
-
Peter Zijlstra authored
objtool to the rescue: vmlinux.o: warning: objtool: intel_idle_ibrs+0x17: call to spec_ctrl_current() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle_ibrs+0x27: call to wrmsrl.constprop.0() leaves .noinstr.text section Signed-off-by:
-
Peter Zijlstra authored
Fix instrumentation bugs objtool found: vmlinux.o: warning: objtool: intel_idle_s2idle+0xd5: call to fpu_idle_fpregs() leaves .noinstr.text section vmlinux.o: warning: objtool: intel_idle_xstate+0x11: call to fpu_idle_fpregs() leaves .noinstr.text section vmlinux.o: warning: objtool: fpu_idle_fpregs+0x9: call to xfeatures_in_use() leaves .noinstr.text section Signed-off-by:
-
Peter Zijlstra authored
So objtool found this bug: vmlinux.o: warning: objtool: intel_idle_irq+0x10c: call to trace_hardirqs_off() leaves .noinstr.text section As per commit 32d4fd57 ("cpuidle,intel_idle: Fix CPUIDLE_FLAG_IRQ_ENABLE"): "must not have tracing in idle functions" Clearly people can't read and tinker along until splat dissapears. This straight up reverts commit d295ad34 ("intel_idle: Fix false positive RCU splats due to incorrect hardirqs state"). It doesn't re-introduce the problem because preceding patches fixed it properly. Fixes: d295ad34 ("intel_idle: Fix false positive RCU splats due to incorrect hardirqs state") Signed-off-by:
-
Peter Zijlstra authored
Idle code is very like entry code in that RCU isn't available. As such, add a little validation. Signed-off-by:
Geert Uytterhoeven <geert@linux-m68k.org> Acked-by:
-
