Commit 01b0f0ea authored by Nicholas Piggin's avatar Nicholas Piggin Committed by Michael Ellerman

powerpc/64s: Trim offlined CPUs from mm_cpumasks

When offlining a CPU, powerpc/64s does not flush TLBs, rather it just
leaves the CPU set in mm_cpumasks, so it continues to receive TLBIEs
to manage its TLBs.

However the exit_flush_lazy_tlbs() function expects that after
returning, all CPUs (except self) have flushed TLBs for that mm, in
which case TLBIEL can be used for this flush. This breaks for offline
CPUs because they don't get the IPI to flush their TLB. This can lead
to stale translations.

Fix this by clearing the CPU from mm_cpumasks, then flushing all TLBs
before going offline.

These offlined CPU bits stuck in the cpumask also prevents the cpumask
from being trimmed back to local mode, which means continual broadcast
IPIs or TLBIEs are needed for TLB flushing. This patch prevents that
situation too.

A cast of many were involved in working this out, but in particular
Milton, Aneesh, Paul made key discoveries.

Fixes: 0cef77c7 ("powerpc/64s/radix: flush remote CPUs out of single-threaded mm_cpumask")
Signed-off-by: default avatarNicholas Piggin <npiggin@gmail.com>
Reviewed-by: default avatarAneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Debugged-by: default avatarMilton Miller <miltonm@us.ibm.com>
Debugged-by: default avatarAneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Debugged-by: default avatarPaul Mackerras <paulus@samba.org>
Signed-off-by: default avatarMichael Ellerman <mpe@ellerman.id.au>
Link: https://lore.kernel.org/r/20201126102530.691335-5-npiggin@gmail.com
parent 8ff00399
......@@ -242,6 +242,18 @@ extern void radix_init_pseries(void);
static inline void radix_init_pseries(void) { };
#endif
#ifdef CONFIG_HOTPLUG_CPU
#define arch_clear_mm_cpumask_cpu(cpu, mm) \
do { \
if (cpumask_test_cpu(cpu, mm_cpumask(mm))) { \
atomic_dec(&(mm)->context.active_cpus); \
cpumask_clear_cpu(cpu, mm_cpumask(mm)); \
} \
} while (0)
void cleanup_cpu_mmu_context(void);
#endif
static inline int get_user_context(mm_context_t *ctx, unsigned long ea)
{
int index = ea >> MAX_EA_BITS_PER_CONTEXT;
......
......@@ -17,6 +17,7 @@
#include <linux/export.h>
#include <linux/gfp.h>
#include <linux/slab.h>
#include <linux/cpu.h>
#include <asm/mmu_context.h>
#include <asm/pgalloc.h>
......@@ -307,3 +308,22 @@ void radix__switch_mmu_context(struct mm_struct *prev, struct mm_struct *next)
isync();
}
#endif
/**
* cleanup_cpu_mmu_context - Clean up MMU details for this CPU (newly offlined)
*
* This clears the CPU from mm_cpumask for all processes, and then flushes the
* local TLB to ensure TLB coherency in case the CPU is onlined again.
*
* KVM guest translations are not necessarily flushed here. If KVM started
* using mm_cpumask or the Linux APIs which do, this would have to be resolved.
*/
#ifdef CONFIG_HOTPLUG_CPU
void cleanup_cpu_mmu_context(void)
{
int cpu = smp_processor_id();
clear_tasks_mm_cpumask(cpu);
tlbiel_all();
}
#endif
......@@ -911,6 +911,8 @@ static int smp_core99_cpu_disable(void)
mpic_cpu_set_priority(0xf);
cleanup_cpu_mmu_context();
return 0;
}
......
......@@ -143,6 +143,9 @@ static int pnv_smp_cpu_disable(void)
xive_smp_disable_cpu();
else
xics_migrate_irqs_away();
cleanup_cpu_mmu_context();
return 0;
}
......
......@@ -90,6 +90,9 @@ static int pseries_cpu_disable(void)
xive_smp_disable_cpu();
else
xics_migrate_irqs_away();
cleanup_cpu_mmu_context();
return 0;
}
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment