Commit f77084d9 authored by Sebastian Andrzej Siewior's avatar Sebastian Andrzej Siewior Committed by Thomas Gleixner

x86/mm/pat: Disable preemption around __flush_tlb_all()

The WARN_ON_ONCE(__read_cr3() != build_cr3()) in switch_mm_irqs_off()
triggers every once in a while during a snapshotted system upgrade.

The warning triggers since commit decab088 ("x86/mm: Remove
preempt_disable/enable() from __native_flush_tlb()"). The callchain is:

  get_page_from_freelist() -> post_alloc_hook() -> __kernel_map_pages()

with CONFIG_DEBUG_PAGEALLOC enabled.

Disable preemption during CR3 reset / __flush_tlb_all() and add a comment
why preemption has to be disabled so it won't be removed accidentaly.

Add another preemptible() check in __flush_tlb_all() to catch callers with
enabled preemption when PGE is enabled, because PGE enabled does not
trigger the warning in __native_flush_tlb(). Suggested by Andy Lutomirski.

Fixes: decab088 ("x86/mm: Remove preempt_disable/enable() from __native_flush_tlb()")
Signed-off-by: default avatarSebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/20181017103432.zgv46nlu3hc7k4rq@linutronix.de
parent 8af19095
...@@ -453,6 +453,12 @@ static inline void __native_flush_tlb_one_user(unsigned long addr) ...@@ -453,6 +453,12 @@ static inline void __native_flush_tlb_one_user(unsigned long addr)
*/ */
static inline void __flush_tlb_all(void) static inline void __flush_tlb_all(void)
{ {
/*
* This is to catch users with enabled preemption and the PGE feature
* and don't trigger the warning in __native_flush_tlb().
*/
VM_WARN_ON_ONCE(preemptible());
if (boot_cpu_has(X86_FEATURE_PGE)) { if (boot_cpu_has(X86_FEATURE_PGE)) {
__flush_tlb_global(); __flush_tlb_global();
} else { } else {
......
...@@ -2309,9 +2309,13 @@ void __kernel_map_pages(struct page *page, int numpages, int enable) ...@@ -2309,9 +2309,13 @@ void __kernel_map_pages(struct page *page, int numpages, int enable)
/* /*
* We should perform an IPI and flush all tlbs, * We should perform an IPI and flush all tlbs,
* but that can deadlock->flush only current cpu: * but that can deadlock->flush only current cpu.
* Preemption needs to be disabled around __flush_tlb_all() due to
* CR3 reload in __native_flush_tlb().
*/ */
preempt_disable();
__flush_tlb_all(); __flush_tlb_all();
preempt_enable();
arch_flush_lazy_mmu_mode(); arch_flush_lazy_mmu_mode();
} }
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment