Commit de56a948 authored by Paul Mackerras's avatar Paul Mackerras Committed by Avi Kivity

KVM: PPC: Add support for Book3S processors in hypervisor mode

This adds support for KVM running on 64-bit Book 3S processors,
specifically POWER7, in hypervisor mode.  Using hypervisor mode means
that the guest can use the processor's supervisor mode.  That means
that the guest can execute privileged instructions and access privileged
registers itself without trapping to the host.  This gives excellent
performance, but does mean that KVM cannot emulate a processor
architecture other than the one that the hardware implements.

This code assumes that the guest is running paravirtualized using the
PAPR (Power Architecture Platform Requirements) interface, which is the
interface that IBM's PowerVM hypervisor uses.  That means that existing
Linux distributions that run on IBM pSeries machines will also run
under KVM without modification.  In order to communicate the PAPR
hypercalls to qemu, this adds a new KVM_EXIT_PAPR_HCALL exit code
to include/linux/kvm.h.

Currently the choice between book3s_hv support and book3s_pr support
(i.e. the existing code, which runs the guest in user mode) has to be
made at kernel configuration time, so a given kernel binary can only
do one or the other.

This new book3s_hv code doesn't support MMIO emulation at present.
Since we are running paravirtualized guests, this isn't a serious
restriction.

With the guest running in supervisor mode, most exceptions go straight
to the guest.  We will never get data or instruction storage or segment
interrupts, alignment interrupts, decrementer interrupts, program
interrupts, single-step interrupts, etc., coming to the hypervisor from
the guest.  Therefore this introduces a new KVMTEST_NONHV macro for the
exception entry path so that we don't have to do the KVM test on entry
to those exception handlers.

We do however get hypervisor decrementer, hypervisor data storage,
hypervisor instruction storage, and hypervisor emulation assist
interrupts, so we have to handle those.

In hypervisor mode, real-mode accesses can access all of RAM, not just
a limited amount.  Therefore we put all the guest state in the vcpu.arch
and use the shadow_vcpu in the PACA only for temporary scratch space.
We allocate the vcpu with kzalloc rather than vzalloc, and we don't use
anything in the kvmppc_vcpu_book3s struct, so we don't allocate it.
We don't have a shared page with the guest, but we still need a
kvm_vcpu_arch_shared struct to store the values of various registers,
so we include one in the vcpu_arch struct.

The POWER7 processor has a restriction that all threads in a core have
to be in the same partition.  MMU-on kernel code counts as a partition
(partition 0), so we have to do a partition switch on every entry to and
exit from the guest.  At present we require the host and guest to run
in single-thread mode because of this hardware restriction.

This code allocates a hashed page table for the guest and initializes
it with HPTEs for the guest's Virtual Real Memory Area (VRMA).  We
require that the guest memory is allocated using 16MB huge pages, in
order to simplify the low-level memory management.  This also means that
we can get away without tracking paging activity in the host for now,
since huge pages can't be paged or swapped.

This also adds a few new exports needed by the book3s_hv code.
Signed-off-by: default avatarPaul Mackerras <paulus@samba.org>
Signed-off-by: default avatarAlexander Graf <agraf@suse.de>
parent 3c42bf8a
...@@ -1532,6 +1532,23 @@ Userspace can now handle the hypercall and when it's done modify the gprs as ...@@ -1532,6 +1532,23 @@ Userspace can now handle the hypercall and when it's done modify the gprs as
necessary. Upon guest entry all guest GPRs will then be replaced by the values necessary. Upon guest entry all guest GPRs will then be replaced by the values
in this struct. in this struct.
/* KVM_EXIT_PAPR_HCALL */
struct {
__u64 nr;
__u64 ret;
__u64 args[9];
} papr_hcall;
This is used on 64-bit PowerPC when emulating a pSeries partition,
e.g. with the 'pseries' machine type in qemu. It occurs when the
guest does a hypercall using the 'sc 1' instruction. The 'nr' field
contains the hypercall number (from the guest R3), and 'args' contains
the arguments (from the guest R4 - R12). Userspace should put the
return code in 'ret' and any extra returned values in args[].
The possible hypercalls are defined in the Power Architecture Platform
Requirements (PAPR) document available from www.power.org (free
developer registration required to access it).
/* Fix the size of the union. */ /* Fix the size of the union. */
char padding[256]; char padding[256];
}; };
......
...@@ -134,6 +134,17 @@ do_kvm_##n: \ ...@@ -134,6 +134,17 @@ do_kvm_##n: \
#define KVM_HANDLER_SKIP(area, h, n) #define KVM_HANDLER_SKIP(area, h, n)
#endif #endif
#ifdef CONFIG_KVM_BOOK3S_PR
#define KVMTEST_PR(n) __KVMTEST(n)
#define KVM_HANDLER_PR(area, h, n) __KVM_HANDLER(area, h, n)
#define KVM_HANDLER_PR_SKIP(area, h, n) __KVM_HANDLER_SKIP(area, h, n)
#else
#define KVMTEST_PR(n)
#define KVM_HANDLER_PR(area, h, n)
#define KVM_HANDLER_PR_SKIP(area, h, n)
#endif
#define NOTEST(n) #define NOTEST(n)
/* /*
...@@ -210,7 +221,7 @@ label##_pSeries: \ ...@@ -210,7 +221,7 @@ label##_pSeries: \
HMT_MEDIUM; \ HMT_MEDIUM; \
SET_SCRATCH0(r13); /* save r13 */ \ SET_SCRATCH0(r13); /* save r13 */ \
EXCEPTION_PROLOG_PSERIES(PACA_EXGEN, label##_common, \ EXCEPTION_PROLOG_PSERIES(PACA_EXGEN, label##_common, \
EXC_STD, KVMTEST, vec) EXC_STD, KVMTEST_PR, vec)
#define STD_EXCEPTION_HV(loc, vec, label) \ #define STD_EXCEPTION_HV(loc, vec, label) \
. = loc; \ . = loc; \
...@@ -227,8 +238,8 @@ label##_hv: \ ...@@ -227,8 +238,8 @@ label##_hv: \
beq masked_##h##interrupt beq masked_##h##interrupt
#define _SOFTEN_TEST(h) __SOFTEN_TEST(h) #define _SOFTEN_TEST(h) __SOFTEN_TEST(h)
#define SOFTEN_TEST(vec) \ #define SOFTEN_TEST_PR(vec) \
KVMTEST(vec); \ KVMTEST_PR(vec); \
_SOFTEN_TEST(EXC_STD) _SOFTEN_TEST(EXC_STD)
#define SOFTEN_TEST_HV(vec) \ #define SOFTEN_TEST_HV(vec) \
...@@ -248,7 +259,7 @@ label##_hv: \ ...@@ -248,7 +259,7 @@ label##_hv: \
.globl label##_pSeries; \ .globl label##_pSeries; \
label##_pSeries: \ label##_pSeries: \
_MASKABLE_EXCEPTION_PSERIES(vec, label, \ _MASKABLE_EXCEPTION_PSERIES(vec, label, \
EXC_STD, SOFTEN_TEST) EXC_STD, SOFTEN_TEST_PR)
#define MASKABLE_EXCEPTION_HV(loc, vec, label) \ #define MASKABLE_EXCEPTION_HV(loc, vec, label) \
. = loc; \ . = loc; \
......
...@@ -64,8 +64,12 @@ ...@@ -64,8 +64,12 @@
#define BOOK3S_INTERRUPT_PROGRAM 0x700 #define BOOK3S_INTERRUPT_PROGRAM 0x700
#define BOOK3S_INTERRUPT_FP_UNAVAIL 0x800 #define BOOK3S_INTERRUPT_FP_UNAVAIL 0x800
#define BOOK3S_INTERRUPT_DECREMENTER 0x900 #define BOOK3S_INTERRUPT_DECREMENTER 0x900
#define BOOK3S_INTERRUPT_HV_DECREMENTER 0x980
#define BOOK3S_INTERRUPT_SYSCALL 0xc00 #define BOOK3S_INTERRUPT_SYSCALL 0xc00
#define BOOK3S_INTERRUPT_TRACE 0xd00 #define BOOK3S_INTERRUPT_TRACE 0xd00
#define BOOK3S_INTERRUPT_H_DATA_STORAGE 0xe00
#define BOOK3S_INTERRUPT_H_INST_STORAGE 0xe20
#define BOOK3S_INTERRUPT_H_EMUL_ASSIST 0xe40
#define BOOK3S_INTERRUPT_PERFMON 0xf00 #define BOOK3S_INTERRUPT_PERFMON 0xf00
#define BOOK3S_INTERRUPT_ALTIVEC 0xf20 #define BOOK3S_INTERRUPT_ALTIVEC 0xf20
#define BOOK3S_INTERRUPT_VSX 0xf40 #define BOOK3S_INTERRUPT_VSX 0xf40
......
...@@ -116,6 +116,7 @@ extern void kvmppc_set_msr(struct kvm_vcpu *vcpu, u64 new_msr); ...@@ -116,6 +116,7 @@ extern void kvmppc_set_msr(struct kvm_vcpu *vcpu, u64 new_msr);
extern void kvmppc_set_pvr(struct kvm_vcpu *vcpu, u32 pvr); extern void kvmppc_set_pvr(struct kvm_vcpu *vcpu, u32 pvr);
extern void kvmppc_mmu_book3s_64_init(struct kvm_vcpu *vcpu); extern void kvmppc_mmu_book3s_64_init(struct kvm_vcpu *vcpu);
extern void kvmppc_mmu_book3s_32_init(struct kvm_vcpu *vcpu); extern void kvmppc_mmu_book3s_32_init(struct kvm_vcpu *vcpu);
extern void kvmppc_mmu_book3s_hv_init(struct kvm_vcpu *vcpu);
extern int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *pte); extern int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *pte);
extern int kvmppc_mmu_map_segment(struct kvm_vcpu *vcpu, ulong eaddr); extern int kvmppc_mmu_map_segment(struct kvm_vcpu *vcpu, ulong eaddr);
extern void kvmppc_mmu_flush_segments(struct kvm_vcpu *vcpu); extern void kvmppc_mmu_flush_segments(struct kvm_vcpu *vcpu);
...@@ -127,10 +128,12 @@ extern int kvmppc_mmu_hpte_init(struct kvm_vcpu *vcpu); ...@@ -127,10 +128,12 @@ extern int kvmppc_mmu_hpte_init(struct kvm_vcpu *vcpu);
extern void kvmppc_mmu_invalidate_pte(struct kvm_vcpu *vcpu, struct hpte_cache *pte); extern void kvmppc_mmu_invalidate_pte(struct kvm_vcpu *vcpu, struct hpte_cache *pte);
extern int kvmppc_mmu_hpte_sysinit(void); extern int kvmppc_mmu_hpte_sysinit(void);
extern void kvmppc_mmu_hpte_sysexit(void); extern void kvmppc_mmu_hpte_sysexit(void);
extern int kvmppc_mmu_hv_init(void);
extern int kvmppc_ld(struct kvm_vcpu *vcpu, ulong *eaddr, int size, void *ptr, bool data); extern int kvmppc_ld(struct kvm_vcpu *vcpu, ulong *eaddr, int size, void *ptr, bool data);
extern int kvmppc_st(struct kvm_vcpu *vcpu, ulong *eaddr, int size, void *ptr, bool data); extern int kvmppc_st(struct kvm_vcpu *vcpu, ulong *eaddr, int size, void *ptr, bool data);
extern void kvmppc_book3s_queue_irqprio(struct kvm_vcpu *vcpu, unsigned int vec); extern void kvmppc_book3s_queue_irqprio(struct kvm_vcpu *vcpu, unsigned int vec);
extern void kvmppc_inject_interrupt(struct kvm_vcpu *vcpu, int vec, u64 flags);
extern void kvmppc_set_bat(struct kvm_vcpu *vcpu, struct kvmppc_bat *bat, extern void kvmppc_set_bat(struct kvm_vcpu *vcpu, struct kvmppc_bat *bat,
bool upper, u32 val); bool upper, u32 val);
extern void kvmppc_giveup_ext(struct kvm_vcpu *vcpu, ulong msr); extern void kvmppc_giveup_ext(struct kvm_vcpu *vcpu, ulong msr);
...@@ -140,6 +143,7 @@ extern pfn_t kvmppc_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn); ...@@ -140,6 +143,7 @@ extern pfn_t kvmppc_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn);
extern void kvmppc_handler_lowmem_trampoline(void); extern void kvmppc_handler_lowmem_trampoline(void);
extern void kvmppc_handler_trampoline_enter(void); extern void kvmppc_handler_trampoline_enter(void);
extern void kvmppc_rmcall(ulong srr0, ulong srr1); extern void kvmppc_rmcall(ulong srr0, ulong srr1);
extern void kvmppc_hv_entry_trampoline(void);
extern void kvmppc_load_up_fpu(void); extern void kvmppc_load_up_fpu(void);
extern void kvmppc_load_up_altivec(void); extern void kvmppc_load_up_altivec(void);
extern void kvmppc_load_up_vsx(void); extern void kvmppc_load_up_vsx(void);
...@@ -151,6 +155,19 @@ static inline struct kvmppc_vcpu_book3s *to_book3s(struct kvm_vcpu *vcpu) ...@@ -151,6 +155,19 @@ static inline struct kvmppc_vcpu_book3s *to_book3s(struct kvm_vcpu *vcpu)
return container_of(vcpu, struct kvmppc_vcpu_book3s, vcpu); return container_of(vcpu, struct kvmppc_vcpu_book3s, vcpu);
} }
extern void kvm_return_point(void);
/* Also add subarch specific defines */
#ifdef CONFIG_KVM_BOOK3S_32_HANDLER
#include <asm/kvm_book3s_32.h>
#endif
#ifdef CONFIG_KVM_BOOK3S_64_HANDLER
#include <asm/kvm_book3s_64.h>
#endif
#ifdef CONFIG_KVM_BOOK3S_PR
static inline unsigned long kvmppc_interrupt_offset(struct kvm_vcpu *vcpu) static inline unsigned long kvmppc_interrupt_offset(struct kvm_vcpu *vcpu)
{ {
return to_book3s(vcpu)->hior; return to_book3s(vcpu)->hior;
...@@ -165,16 +182,6 @@ static inline void kvmppc_update_int_pending(struct kvm_vcpu *vcpu, ...@@ -165,16 +182,6 @@ static inline void kvmppc_update_int_pending(struct kvm_vcpu *vcpu,
vcpu->arch.shared->int_pending = 0; vcpu->arch.shared->int_pending = 0;
} }
static inline ulong dsisr(void)
{
ulong r;
asm ( "mfdsisr %0 " : "=r" (r) );
return r;
}
extern void kvm_return_point(void);
static inline struct kvmppc_book3s_shadow_vcpu *to_svcpu(struct kvm_vcpu *vcpu);
static inline void kvmppc_set_gpr(struct kvm_vcpu *vcpu, int num, ulong val) static inline void kvmppc_set_gpr(struct kvm_vcpu *vcpu, int num, ulong val)
{ {
if ( num < 14 ) { if ( num < 14 ) {
...@@ -281,6 +288,108 @@ static inline bool kvmppc_critical_section(struct kvm_vcpu *vcpu) ...@@ -281,6 +288,108 @@ static inline bool kvmppc_critical_section(struct kvm_vcpu *vcpu)
return crit; return crit;
} }
#else /* CONFIG_KVM_BOOK3S_PR */
static inline unsigned long kvmppc_interrupt_offset(struct kvm_vcpu *vcpu)
{
return 0;
}
static inline void kvmppc_update_int_pending(struct kvm_vcpu *vcpu,
unsigned long pending_now, unsigned long old_pending)
{
/* Recalculate LPCR:MER based on the presence of
* a pending external interrupt
*/
if (test_bit(BOOK3S_IRQPRIO_EXTERNAL, &pending_now) ||
test_bit(BOOK3S_IRQPRIO_EXTERNAL_LEVEL, &pending_now))
vcpu->arch.lpcr |= LPCR_MER;
else
vcpu->arch.lpcr &= ~((u64)LPCR_MER);
}
static inline void kvmppc_set_gpr(struct kvm_vcpu *vcpu, int num, ulong val)
{
vcpu->arch.gpr[num] = val;
}
static inline ulong kvmppc_get_gpr(struct kvm_vcpu *vcpu, int num)
{
return vcpu->arch.gpr[num];
}
static inline void kvmppc_set_cr(struct kvm_vcpu *vcpu, u32 val)
{
vcpu->arch.cr = val;
}
static inline u32 kvmppc_get_cr(struct kvm_vcpu *vcpu)
{
return vcpu->arch.cr;
}
static inline void kvmppc_set_xer(struct kvm_vcpu *vcpu, u32 val)
{
vcpu->arch.xer = val;
}
static inline u32 kvmppc_get_xer(struct kvm_vcpu *vcpu)
{
return vcpu->arch.xer;
}
static inline void kvmppc_set_ctr(struct kvm_vcpu *vcpu, ulong val)
{
vcpu->arch.ctr = val;
}
static inline ulong kvmppc_get_ctr(struct kvm_vcpu *vcpu)
{
return vcpu->arch.ctr;
}
static inline void kvmppc_set_lr(struct kvm_vcpu *vcpu, ulong val)
{
vcpu->arch.lr = val;
}
static inline ulong kvmppc_get_lr(struct kvm_vcpu *vcpu)
{
return vcpu->arch.lr;
}
static inline void kvmppc_set_pc(struct kvm_vcpu *vcpu, ulong val)
{
vcpu->arch.pc = val;
}
static inline ulong kvmppc_get_pc(struct kvm_vcpu *vcpu)
{
return vcpu->arch.pc;
}
static inline u32 kvmppc_get_last_inst(struct kvm_vcpu *vcpu)
{
ulong pc = kvmppc_get_pc(vcpu);
/* Load the instruction manually if it failed to do so in the
* exit path */
if (vcpu->arch.last_inst == KVM_INST_FETCH_FAILED)
kvmppc_ld(vcpu, &pc, sizeof(u32), &vcpu->arch.last_inst, false);
return vcpu->arch.last_inst;
}
static inline ulong kvmppc_get_fault_dar(struct kvm_vcpu *vcpu)
{
return vcpu->arch.fault_dar;
}
static inline bool kvmppc_critical_section(struct kvm_vcpu *vcpu)
{
return false;
}
#endif
/* Magic register values loaded into r3 and r4 before the 'sc' assembly /* Magic register values loaded into r3 and r4 before the 'sc' assembly
* instruction for the OSI hypercalls */ * instruction for the OSI hypercalls */
...@@ -289,12 +398,4 @@ static inline bool kvmppc_critical_section(struct kvm_vcpu *vcpu) ...@@ -289,12 +398,4 @@ static inline bool kvmppc_critical_section(struct kvm_vcpu *vcpu)
#define INS_DCBZ 0x7c0007ec #define INS_DCBZ 0x7c0007ec
/* Also add subarch specific defines */
#ifdef CONFIG_PPC_BOOK3S_32
#include <asm/kvm_book3s_32.h>
#else
#include <asm/kvm_book3s_64.h>
#endif
#endif /* __ASM_KVM_BOOK3S_H__ */ #endif /* __ASM_KVM_BOOK3S_H__ */
...@@ -20,9 +20,11 @@ ...@@ -20,9 +20,11 @@
#ifndef __ASM_KVM_BOOK3S_64_H__ #ifndef __ASM_KVM_BOOK3S_64_H__
#define __ASM_KVM_BOOK3S_64_H__ #define __ASM_KVM_BOOK3S_64_H__
#ifdef CONFIG_KVM_BOOK3S_PR
static inline struct kvmppc_book3s_shadow_vcpu *to_svcpu(struct kvm_vcpu *vcpu) static inline struct kvmppc_book3s_shadow_vcpu *to_svcpu(struct kvm_vcpu *vcpu)
{ {
return &get_paca()->shadow_vcpu; return &get_paca()->shadow_vcpu;
} }
#endif
#endif /* __ASM_KVM_BOOK3S_64_H__ */ #endif /* __ASM_KVM_BOOK3S_64_H__ */
...@@ -70,10 +70,22 @@ kvmppc_resume_\intno: ...@@ -70,10 +70,22 @@ kvmppc_resume_\intno:
struct kvmppc_host_state { struct kvmppc_host_state {
ulong host_r1; ulong host_r1;
ulong host_r2; ulong host_r2;
ulong host_msr;
ulong vmhandler; ulong vmhandler;
ulong scratch0; ulong scratch0;
ulong scratch1; ulong scratch1;
u8 in_guest; u8 in_guest;
#ifdef CONFIG_KVM_BOOK3S_64_HV
struct kvm_vcpu *kvm_vcpu;
u64 dabr;
u64 host_mmcr[3];
u32 host_pmc[6];
u64 host_purr;
u64 host_spurr;
u64 host_dscr;
u64 dec_expires;
#endif
}; };
struct kvmppc_book3s_shadow_vcpu { struct kvmppc_book3s_shadow_vcpu {
......
...@@ -93,4 +93,8 @@ static inline ulong kvmppc_get_fault_dar(struct kvm_vcpu *vcpu) ...@@ -93,4 +93,8 @@ static inline ulong kvmppc_get_fault_dar(struct kvm_vcpu *vcpu)
return vcpu->arch.fault_dear; return vcpu->arch.fault_dear;
} }
static inline ulong kvmppc_get_msr(struct kvm_vcpu *vcpu)
{
return vcpu->arch.shared->msr;
}
#endif /* __ASM_KVM_BOOKE_H__ */ #endif /* __ASM_KVM_BOOKE_H__ */
...@@ -33,7 +33,9 @@ ...@@ -33,7 +33,9 @@
/* memory slots that does not exposed to userspace */ /* memory slots that does not exposed to userspace */
#define KVM_PRIVATE_MEM_SLOTS 4 #define KVM_PRIVATE_MEM_SLOTS 4
#ifdef CONFIG_KVM_MMIO
#define KVM_COALESCED_MMIO_PAGE_OFFSET 1 #define KVM_COALESCED_MMIO_PAGE_OFFSET 1
#endif
/* We don't currently support large pages. */ /* We don't currently support large pages. */
#define KVM_HPAGE_GFN_SHIFT(x) 0 #define KVM_HPAGE_GFN_SHIFT(x) 0
...@@ -133,7 +135,26 @@ struct kvmppc_exit_timing { ...@@ -133,7 +135,26 @@ struct kvmppc_exit_timing {
}; };
}; };
struct kvmppc_pginfo {
unsigned long pfn;
atomic_t refcnt;
};
struct kvm_arch { struct kvm_arch {
#ifdef CONFIG_KVM_BOOK3S_64_HV
unsigned long hpt_virt;
unsigned long ram_npages;
unsigned long ram_psize;
unsigned long ram_porder;
struct kvmppc_pginfo *ram_pginfo;
unsigned int lpid;
unsigned int host_lpid;
unsigned long host_lpcr;
unsigned long sdr1;
unsigned long host_sdr1;
int tlbie_lock;
unsigned short last_vcpu[NR_CPUS];
#endif /* CONFIG_KVM_BOOK3S_64_HV */
}; };
struct kvmppc_pte { struct kvmppc_pte {
...@@ -190,7 +211,7 @@ struct kvm_vcpu_arch { ...@@ -190,7 +211,7 @@ struct kvm_vcpu_arch {
ulong rmcall; ulong rmcall;
ulong host_paca_phys; ulong host_paca_phys;
struct kvmppc_slb slb[64]; struct kvmppc_slb slb[64];
int slb_max; /* # valid entries in slb[] */ int slb_max; /* 1 + index of last valid entry in slb[] */
int slb_nr; /* total number of entries in SLB */ int slb_nr; /* total number of entries in SLB */
struct kvmppc_mmu mmu; struct kvmppc_mmu mmu;
#endif #endif
...@@ -212,7 +233,7 @@ struct kvm_vcpu_arch { ...@@ -212,7 +233,7 @@ struct kvm_vcpu_arch {
#endif #endif
#ifdef CONFIG_VSX #ifdef CONFIG_VSX
u64 vsr[32]; u64 vsr[64];
#endif #endif
#ifdef CONFIG_PPC_BOOK3S #ifdef CONFIG_PPC_BOOK3S
...@@ -220,18 +241,24 @@ struct kvm_vcpu_arch { ...@@ -220,18 +241,24 @@ struct kvm_vcpu_arch {
u32 qpr[32]; u32 qpr[32];
#endif #endif
#ifdef CONFIG_BOOKE
ulong pc; ulong pc;
ulong ctr; ulong ctr;
ulong lr; ulong lr;
ulong xer; ulong xer;
u32 cr; u32 cr;
#endif
#ifdef CONFIG_PPC_BOOK3S #ifdef CONFIG_PPC_BOOK3S
ulong hflags; ulong hflags;
ulong guest_owned_ext; ulong guest_owned_ext;
ulong purr;
ulong spurr;
ulong lpcr;
ulong dscr;
ulong amr;
ulong uamor;
u32 ctrl;
ulong dabr;
#endif #endif
u32 vrsave; /* also USPRG0 */ u32 vrsave; /* also USPRG0 */
u32 mmucr; u32 mmucr;
...@@ -270,6 +297,9 @@ struct kvm_vcpu_arch { ...@@ -270,6 +297,9 @@ struct kvm_vcpu_arch {
u32 dbcr1; u32 dbcr1;
u32 dbsr; u32 dbsr;
u64 mmcr[3];
u32 pmc[6];
#ifdef CONFIG_KVM_EXIT_TIMING #ifdef CONFIG_KVM_EXIT_TIMING
struct mutex exit_timing_lock; struct mutex exit_timing_lock;
struct kvmppc_exit_timing timing_exit; struct kvmppc_exit_timing timing_exit;
...@@ -284,8 +314,12 @@ struct kvm_vcpu_arch { ...@@ -284,8 +314,12 @@ struct kvm_vcpu_arch {
struct dentry *debugfs_exit_timing; struct dentry *debugfs_exit_timing;
#endif #endif
#ifdef CONFIG_PPC_BOOK3S
ulong fault_dar;
u32 fault_dsisr;
#endif
#ifdef CONFIG_BOOKE #ifdef CONFIG_BOOKE
u32 last_inst;
ulong fault_dear; ulong fault_dear;
ulong fault_esr; ulong fault_esr;
ulong queued_dear; ulong queued_dear;
...@@ -300,16 +334,25 @@ struct kvm_vcpu_arch { ...@@ -300,16 +334,25 @@ struct kvm_vcpu_arch {
u8 dcr_is_write; u8 dcr_is_write;
u8 osi_needed; u8 osi_needed;
u8 osi_enabled; u8 osi_enabled;
u8 hcall_needed;
u32 cpr0_cfgaddr; /* holds the last set cpr0_cfgaddr */ u32 cpr0_cfgaddr; /* holds the last set cpr0_cfgaddr */
struct hrtimer dec_timer; struct hrtimer dec_timer;
struct tasklet_struct tasklet; struct tasklet_struct tasklet;
u64 dec_jiffies; u64 dec_jiffies;
u64 dec_expires;
unsigned long pending_exceptions; unsigned long pending_exceptions;
u16 last_cpu;
u32 last_inst;
int trap;
struct kvm_vcpu_arch_shared *shared; struct kvm_vcpu_arch_shared *shared;
unsigned long magic_page_pa; /* phys addr to map the magic page to */ unsigned long magic_page_pa; /* phys addr to map the magic page to */
unsigned long magic_page_ea; /* effect. addr to map the magic page to */ unsigned long magic_page_ea; /* effect. addr to map the magic page to */
#ifdef CONFIG_KVM_BOOK3S_64_HV
struct kvm_vcpu_arch_shared shregs;
#endif
}; };
#endif /* __POWERPC_KVM_HOST_H__ */ #endif /* __POWERPC_KVM_HOST_H__ */
...@@ -112,6 +112,12 @@ extern void kvmppc_core_destroy_mmu(struct kvm_vcpu *vcpu); ...@@ -112,6 +112,12 @@ extern void kvmppc_core_destroy_mmu(struct kvm_vcpu *vcpu);
extern int kvmppc_kvm_pv(struct kvm_vcpu *vcpu); extern int kvmppc_kvm_pv(struct kvm_vcpu *vcpu);
extern void kvmppc_map_magic(struct kvm_vcpu *vcpu); extern void kvmppc_map_magic(struct kvm_vcpu *vcpu);
extern long kvmppc_alloc_hpt(struct kvm *kvm);
extern void kvmppc_free_hpt(struct kvm *kvm);
extern long kvmppc_prepare_vrma(struct kvm *kvm,
struct kvm_userspace_memory_region *mem);
extern void kvmppc_map_vrma(struct kvm *kvm,
struct kvm_userspace_memory_region *mem);
extern int kvmppc_core_init_vm(struct kvm *kvm); extern int kvmppc_core_init_vm(struct kvm *kvm);
extern void kvmppc_core_destroy_vm(struct kvm *kvm); extern void kvmppc_core_destroy_vm(struct kvm *kvm);
extern int kvmppc_core_prepare_memory_region(struct kvm *kvm, extern int kvmppc_core_prepare_memory_region(struct kvm *kvm,
......
...@@ -90,13 +90,19 @@ extern char initial_stab[]; ...@@ -90,13 +90,19 @@ extern char initial_stab[];
#define HPTE_R_PP0 ASM_CONST(0x8000000000000000) #define HPTE_R_PP0 ASM_CONST(0x8000000000000000)
#define HPTE_R_TS ASM_CONST(0x4000000000000000) #define HPTE_R_TS ASM_CONST(0x4000000000000000)
#define HPTE_R_KEY_HI ASM_CONST(0x3000000000000000)
#define HPTE_R_RPN_SHIFT 12 #define HPTE_R_RPN_SHIFT 12
#define HPTE_R_RPN ASM_CONST(0x3ffffffffffff000) #define HPTE_R_RPN ASM_CONST(0x0ffffffffffff000)
#define HPTE_R_FLAGS ASM_CONST(0x00000000000003ff)
#define HPTE_R_PP ASM_CONST(0x0000000000000003) #define HPTE_R_PP ASM_CONST(0x0000000000000003)
#define HPTE_R_N ASM_CONST(0x0000000000000004) #define HPTE_R_N ASM_CONST(0x0000000000000004)
#define HPTE_R_G ASM_CONST(0x0000000000000008)
#define HPTE_R_M ASM_CONST(0x0000000000000010)
#define HPTE_R_I ASM_CONST(0x0000000000000020)
#define HPTE_R_W ASM_CONST(0x0000000000000040)
#define HPTE_R_WIMG ASM_CONST(0x0000000000000078)
#define HPTE_R_C ASM_CONST(0x0000000000000080) #define HPTE_R_C ASM_CONST(0x0000000000000080)
#define HPTE_R_R ASM_CONST(0x0000000000000100) #define HPTE_R_R ASM_CONST(0x0000000000000100)
#define HPTE_R_KEY_LO ASM_CONST(0x0000000000000e00)
#define HPTE_V_1TB_SEG ASM_CONST(0x4000000000000000) #define HPTE_V_1TB_SEG ASM_CONST(0x4000000000000000)
#define HPTE_V_VRMA_MASK ASM_CONST(0x4001ffffff000000) #define HPTE_V_VRMA_MASK ASM_CONST(0x4001ffffff000000)
......
...@@ -147,8 +147,10 @@ struct paca_struct { ...@@ -147,8 +147,10 @@ struct paca_struct {
struct dtl_entry *dtl_curr; /* pointer corresponding to dtl_ridx */ struct dtl_entry *dtl_curr; /* pointer corresponding to dtl_ridx */
#ifdef CONFIG_KVM_BOOK3S_HANDLER #ifdef CONFIG_KVM_BOOK3S_HANDLER
#ifdef CONFIG_KVM_BOOK3S_PR
/* We use this to store guest state in */ /* We use this to store guest state in */
struct kvmppc_book3s_shadow_vcpu shadow_vcpu; struct kvmppc_book3s_shadow_vcpu shadow_vcpu;
#endif
struct kvmppc_host_state kvm_hstate; struct kvmppc_host_state kvm_hstate;
#endif #endif
}; };
......
...@@ -189,6 +189,9 @@ ...@@ -189,6 +189,9 @@
#define SPRN_CTR 0x009 /* Count Register */ #define SPRN_CTR 0x009 /* Count Register */
#define SPRN_DSCR 0x11 #define SPRN_DSCR 0x11
#define SPRN_CFAR 0x1c /* Come From Address Register */ #define SPRN_CFAR 0x1c /* Come From Address Register */
#define SPRN_AMR 0x1d /* Authority Mask Register */
#define SPRN_UAMOR 0x9d /* User Authority Mask Override Register */
#define SPRN_AMOR 0x15d /* Authority Mask Override Register */
#define SPRN_ACOP 0x1F /* Available Coprocessor Register */ #define SPRN_ACOP 0x1F /* Available Coprocessor Register */
#define SPRN_CTRLF 0x088 #define SPRN_CTRLF 0x088
#define SPRN_CTRLT 0x098 #define SPRN_CTRLT 0x098
...@@ -252,6 +255,7 @@ ...@@ -252,6 +255,7 @@
#define LPCR_RMI 0x00000002 /* real mode is cache inhibit */ #define LPCR_RMI 0x00000002 /* real mode is cache inhibit */
#define LPCR_HDICE 0x00000001 /* Hyp Decr enable (HV,PR,EE) */ #define LPCR_HDICE 0x00000001 /* Hyp Decr enable (HV,PR,EE) */
#define SPRN_LPID 0x13F /* Logical Partition Identifier */ #define SPRN_LPID 0x13F /* Logical Partition Identifier */
#define LPID_RSVD 0x3ff /* Reserved LPID for partn switching */
#define SPRN_HMER 0x150 /* Hardware m? error recovery */ #define SPRN_HMER 0x150 /* Hardware m? error recovery */
#define SPRN_HMEER 0x151 /* Hardware m? enable error recovery */ #define SPRN_HMEER 0x151 /* Hardware m? enable error recovery */
#define SPRN_HEIR 0x153 /* Hypervisor Emulated Instruction Register */ #define SPRN_HEIR 0x153 /* Hypervisor Emulated Instruction Register */
......
...@@ -187,6 +187,7 @@ int main(void) ...@@ -187,6 +187,7 @@ int main(void)
DEFINE(LPPACASRR1, offsetof(struct lppaca, saved_srr1)); DEFINE(LPPACASRR1, offsetof(struct lppaca, saved_srr1));
DEFINE(LPPACAANYINT, offsetof(struct lppaca, int_dword.any_int)); DEFINE(LPPACAANYINT, offsetof(struct lppaca, int_dword.any_int));
DEFINE(LPPACADECRINT, offsetof(struct lppaca, int_dword.fields.decr_int)); DEFINE(LPPACADECRINT, offsetof(struct lppaca, int_dword.fields.decr_int));
DEFINE(LPPACA_PMCINUSE, offsetof(struct lppaca, pmcregs_in_use));
DEFINE(LPPACA_DTLIDX, offsetof(struct lppaca, dtl_idx)); DEFINE(LPPACA_DTLIDX, offsetof(struct lppaca, dtl_idx));
DEFINE(PACA_DTL_RIDX, offsetof(struct paca_struct, dtl_ridx)); DEFINE(PACA_DTL_RIDX, offsetof(struct paca_struct, dtl_ridx));
#endif /* CONFIG_PPC_STD_MMU_64 */ #endif /* CONFIG_PPC_STD_MMU_64 */
...@@ -392,6 +393,29 @@ int main(void) ...@@ -392,6 +393,29 @@ int main(void)
DEFINE(VCPU_HOST_PID, offsetof(struct kvm_vcpu, arch.host_pid)); DEFINE(VCPU_HOST_PID, offsetof(struct kvm_vcpu, arch.host_pid));
DEFINE(VCPU_GPRS, offsetof(struct kvm_vcpu, arch.gpr)); DEFINE(VCPU_GPRS, offsetof(struct kvm_vcpu, arch.gpr));
DEFINE(VCPU_VRSAVE, offsetof(struct kvm_vcpu, arch.vrsave)); DEFINE(VCPU_VRSAVE, offsetof(struct kvm_vcpu, arch.vrsave));
DEFINE(VCPU_FPRS, offsetof(struct kvm_vcpu, arch.fpr));
DEFINE(VCPU_FPSCR, offsetof(struct kvm_vcpu, arch.fpscr));
#ifdef CONFIG_ALTIVEC
DEFINE(VCPU_VRS, offsetof(struct kvm_vcpu, arch.vr));
DEFINE(VCPU_VSCR, offsetof(struct kvm_vcpu, arch.vscr));
#endif
#ifdef CONFIG_VSX
DEFINE(VCPU_VSRS, offsetof(struct kvm_vcpu, arch.vsr));
#endif
DEFINE(VCPU_XER, offsetof(struct kvm_vcpu, arch.xer));
DEFINE(VCPU_CTR, offsetof(struct kvm_vcpu, arch.ctr));
DEFINE(VCPU_LR, offsetof(struct kvm_vcpu, arch.lr));
DEFINE(VCPU_CR, offsetof(struct kvm_vcpu, arch.cr));
DEFINE(VCPU_PC, offsetof(struct kvm_vcpu, arch.pc));
#ifdef CONFIG_KVM_BOOK3S_64_HV
DEFINE(VCPU_MSR, offsetof(struct kvm_vcpu, arch.shregs.msr));
DEFINE(VCPU_SRR0, offsetof(struct kvm_vcpu, arch.shregs.srr0));
DEFINE(VCPU_SRR1, offsetof(struct kvm_vcpu, arch.shregs.srr1));
DEFINE(VCPU_SPRG0, offsetof(struct kvm_vcpu, arch.shregs.sprg0));
DEFINE(VCPU_SPRG1, offsetof(struct kvm_vcpu, arch.shregs.sprg1));
DEFINE(VCPU_SPRG2, offsetof(struct kvm_vcpu, arch.shregs.sprg2));
DEFINE(VCPU_SPRG3, offsetof(struct kvm_vcpu, arch.shregs.sprg3));
#endif
DEFINE(VCPU_SPRG4, offsetof(struct kvm_vcpu, arch.sprg4)); DEFINE(VCPU_SPRG4, offsetof(struct kvm_vcpu, arch.sprg4));
DEFINE(VCPU_SPRG5, offsetof(struct kvm_vcpu, arch.sprg5)); DEFINE(VCPU_SPRG5, offsetof(struct kvm_vcpu, arch.sprg5));
DEFINE(VCPU_SPRG6, offsetof(struct kvm_vcpu, arch.sprg6)); DEFINE(VCPU_SPRG6, offsetof(struct kvm_vcpu, arch.sprg6));
...@@ -403,17 +427,60 @@ int main(void) ...@@ -403,17 +427,60 @@ int main(void)
DEFINE(VCPU_SHADOW_MSR, offsetof(struct kvm_vcpu, arch.shadow_msr)); DEFINE(VCPU_SHADOW_MSR, offsetof(struct kvm_vcpu, arch.shadow_msr));
/* book3s */ /* book3s */
#ifdef CONFIG_KVM_BOOK3S_64_HV
DEFINE(KVM_LPID, offsetof(struct kvm, arch.lpid));
DEFINE(KVM_SDR1, offsetof(struct kvm, arch.sdr1));
DEFINE(KVM_HOST_LPID, offsetof(struct kvm, arch.host_lpid));
DEFINE(KVM_HOST_LPCR, offsetof(struct kvm, arch.host_lpcr));
DEFINE(KVM_HOST_SDR1, offsetof(struct kvm, arch.host_sdr1));
DEFINE(KVM_TLBIE_LOCK, offsetof(struct kvm, arch.tlbie_lock));
DEFINE(KVM_ONLINE_CPUS, offsetof(struct kvm, online_vcpus.counter));
DEFINE(KVM_LAST_VCPU, offsetof(struct kvm, arch.last_vcpu));
DEFINE(VCPU_DSISR, offsetof(struct kvm_vcpu, arch.shregs.dsisr));
DEFINE(VCPU_DAR, offsetof(struct kvm_vcpu, arch.shregs.dar));
#endif
#ifdef CONFIG_PPC_BOOK3S #ifdef CONFIG_PPC_BOOK3S
DEFINE(VCPU_KVM, offsetof(struct kvm_vcpu, kvm));
DEFINE(VCPU_VCPUID, offsetof(struct kvm_vcpu, vcpu_id));
DEFINE(VCPU_HOST_RETIP, offsetof(struct kvm_vcpu, arch.host_retip)); DEFINE(VCPU_HOST_RETIP, offsetof(struct kvm_vcpu, arch.host_retip));
DEFINE(VCPU_HOST_MSR, offsetof(struct kvm_vcpu, arch.host_msr)); DEFINE(VCPU_HOST_MSR, offsetof(struct kvm_vcpu, arch.host_msr));
DEFINE(VCPU_PURR, offsetof(struct kvm_vcpu, arch.purr));
DEFINE(VCPU_SPURR, offsetof(struct kvm_vcpu, arch.spurr));
DEFINE(VCPU_DSCR, offsetof(struct kvm_vcpu, arch.dscr));
DEFINE(VCPU_AMR, offsetof(struct kvm_vcpu, arch.amr));
DEFINE(VCPU_UAMOR, offsetof(struct kvm_vcpu, arch.uamor));
DEFINE(VCPU_CTRL, offsetof(struct kvm_vcpu, arch.ctrl));
DEFINE(VCPU_DABR, offsetof(struct kvm_vcpu, arch.dabr));
DEFINE(VCPU_TRAMPOLINE_LOWMEM, offsetof(struct kvm_vcpu, arch.trampoline_lowmem)); DEFINE(VCPU_TRAMPOLINE_LOWMEM, offsetof(struct kvm_vcpu, arch.trampoline_lowmem));
DEFINE(VCPU_TRAMPOLINE_ENTER, offsetof(struct kvm_vcpu, arch.trampoline_enter)); DEFINE(VCPU_TRAMPOLINE_ENTER, offsetof(struct kvm_vcpu, arch.trampoline_enter));
DEFINE(VCPU_HIGHMEM_HANDLER, offsetof(struct kvm_vcpu, arch.highmem_handler)); DEFINE(VCPU_HIGHMEM_HANDLER, offsetof(struct kvm_vcpu, arch.highmem_handler));
DEFINE(VCPU_RMCALL, offsetof(struct kvm_vcpu, arch.rmcall)); DEFINE(VCPU_RMCALL, offsetof(struct kvm_vcpu, arch.rmcall));
DEFINE(VCPU_HFLAGS, offsetof(struct kvm_vcpu, arch.hflags)); DEFINE(VCPU_HFLAGS, offsetof(struct kvm_vcpu, arch.hflags));
DEFINE(VCPU_DEC, offsetof(struct kvm_vcpu, arch.dec));
DEFINE(VCPU_DEC_EXPIRES, offsetof(struct kvm_vcpu, arch.dec_expires));
DEFINE(VCPU_LPCR, offsetof(struct kvm_vcpu, arch.lpcr));
DEFINE(VCPU_MMCR, offsetof(struct kvm_vcpu, arch.mmcr));
DEFINE(VCPU_PMC, offsetof(struct kvm_vcpu, arch.pmc));
DEFINE(VCPU_SLB, offsetof(struct kvm_vcpu, arch.slb));
DEFINE(VCPU_SLB_MAX, offsetof(struct kvm_vcpu, arch.slb_max));
DEFINE(VCPU_SLB_NR, offsetof(struct kvm_vcpu, arch.slb_nr));
DEFINE(VCPU_LAST_CPU, offsetof(struct kvm_vcpu, arch.last_cpu));
DEFINE(VCPU_FAULT_DSISR, offsetof(struct kvm_vcpu, arch.fault_dsisr));
DEFINE(VCPU_FAULT_DAR, offsetof(struct kvm_vcpu, arch.fault_dar));
DEFINE(VCPU_LAST_INST, offsetof(struct kvm_vcpu, arch.last_inst));
DEFINE(VCPU_TRAP, offsetof(struct kvm_vcpu, arch.trap));
DEFINE(VCPU_SVCPU, offsetof(struct kvmppc_vcpu_book3s, shadow_vcpu) -
offsetof(struct kvmppc_vcpu_book3s, vcpu));
DEFINE(VCPU_SLB_E, offsetof(struct kvmppc_slb, orige));
DEFINE(VCPU_SLB_V, offsetof(struct kvmppc_slb, origv));
DEFINE(VCPU_SLB_SIZE, sizeof(struct kvmppc_slb));
#ifdef CONFIG_PPC_BOOK3S_64 #ifdef CONFIG_PPC_BOOK3S_64
#ifdef CONFIG_KVM_BOOK3S_PR
# define SVCPU_FIELD(x, f) DEFINE(x, offsetof(struct paca_struct, shadow_vcpu.f)) # define SVCPU_FIELD(x, f) DEFINE(x, offsetof(struct paca_struct, shadow_vcpu.f))
#else
# define SVCPU_FIELD(x, f)
#endif
# define HSTATE_FIELD(x, f) DEFINE(x, offsetof(struct paca_struct, kvm_hstate.f)) # define HSTATE_FIELD(x, f) DEFINE(x, offsetof(struct paca_struct, kvm_hstate.f))
#else /* 32-bit */ #else /* 32-bit */
# define SVCPU_FIELD(x, f) DEFINE(x, offsetof(struct kvmppc_book3s_shadow_vcpu, f)) # define SVCPU_FIELD(x, f) DEFINE(x, offsetof(struct kvmppc_book3s_shadow_vcpu, f))
...@@ -453,11 +520,23 @@ int main(void) ...@@ -453,11 +520,23 @@ int main(void)
HSTATE_FIELD(HSTATE_HOST_R1, host_r1); HSTATE_FIELD(HSTATE_HOST_R1, host_r1);
HSTATE_FIELD(HSTATE_HOST_R2, host_r2); HSTATE_FIELD(HSTATE_HOST_R2, host_r2);
HSTATE_FIELD(HSTATE_HOST_MSR, host_msr);
HSTATE_FIELD(HSTATE_VMHANDLER, vmhandler); HSTATE_FIELD(HSTATE_VMHANDLER, vmhandler);
HSTATE_FIELD(HSTATE_SCRATCH0, scratch0); HSTATE_FIELD(HSTATE_SCRATCH0, scratch0);
HSTATE_FIELD(HSTATE_SCRATCH1, scratch1); HSTATE_FIELD(HSTATE_SCRATCH1, scratch1);
HSTATE_FIELD(HSTATE_IN_GUEST, in_guest); HSTATE_FIELD(HSTATE_IN_GUEST, in_guest);
#ifdef CONFIG_KVM_BOOK3S_64_HV
HSTATE_FIELD(HSTATE_KVM_VCPU, kvm_vcpu);
HSTATE_FIELD(HSTATE_MMCR, host_mmcr);
HSTATE_FIELD(HSTATE_PMC, host_pmc);
HSTATE_FIELD(HSTATE_PURR, host_purr);
HSTATE_FIELD(HSTATE_SPURR, host_spurr);
HSTATE_FIELD(HSTATE_DSCR, host_dscr);
HSTATE_FIELD(HSTATE_DABR, dabr);
HSTATE_FIELD(HSTATE_DECEXP, dec_expires);
#endif /* CONFIG_KVM_BOOK3S_64_HV */
#else /* CONFIG_PPC_BOOK3S */ #else /* CONFIG_PPC_BOOK3S */
DEFINE(VCPU_CR, offsetof(struct kvm_vcpu, arch.cr)); DEFINE(VCPU_CR, offsetof(struct kvm_vcpu, arch.cr));
DEFINE(VCPU_XER, offsetof(struct kvm_vcpu, arch.xer)); DEFINE(VCPU_XER, offsetof(struct kvm_vcpu, arch.xer));
......
...@@ -87,14 +87,14 @@ data_access_not_stab: ...@@ -87,14 +87,14 @@ data_access_not_stab:
END_MMU_FTR_SECTION_IFCLR(MMU_FTR_SLB) END_MMU_FTR_SECTION_IFCLR(MMU_FTR_SLB)
#endif #endif
EXCEPTION_PROLOG_PSERIES(PACA_EXGEN, data_access_common, EXC_STD, EXCEPTION_PROLOG_PSERIES(PACA_EXGEN, data_access_common, EXC_STD,
KVMTEST, 0x300) KVMTEST_PR, 0x300)
. = 0x380 . = 0x380
.globl data_access_slb_pSeries .globl data_access_slb_pSeries
data_access_slb_pSeries: data_access_slb_pSeries:
HMT_MEDIUM HMT_MEDIUM
SET_SCRATCH0(r13) SET_SCRATCH0(r13)
EXCEPTION_PROLOG_1(PACA_EXSLB, KVMTEST, 0x380) EXCEPTION_PROLOG_1(PACA_EXSLB, KVMTEST_PR, 0x380)
std r3,PACA_EXSLB+EX_R3(r13) std r3,PACA_EXSLB+EX_R3(r13)
mfspr r3,SPRN_DAR mfspr r3,SPRN_DAR
#ifdef __DISABLED__ #ifdef __DISABLED__
...@@ -125,7 +125,7 @@ data_access_slb_pSeries: ...@@ -125,7 +125,7 @@ data_access_slb_pSeries:
instruction_access_slb_pSeries: instruction_access_slb_pSeries:
HMT_MEDIUM HMT_MEDIUM
SET_SCRATCH0(r13) SET_SCRATCH0(r13)
EXCEPTION_PROLOG_1(PACA_EXSLB, KVMTEST, 0x480) EXCEPTION_PROLOG_1(PACA_EXSLB, KVMTEST_PR, 0x480)
std r3,PACA_EXSLB+EX_R3(r13) std r3,PACA_EXSLB+EX_R3(r13)
mfspr r3,SPRN_SRR0 /* SRR0 is faulting address */ mfspr r3,SPRN_SRR0 /* SRR0 is faulting address */
#ifdef __DISABLED__ #ifdef __DISABLED__
...@@ -153,32 +153,32 @@ instruction_access_slb_pSeries: ...@@ -153,32 +153,32 @@ instruction_access_slb_pSeries:
hardware_interrupt_pSeries: hardware_interrupt_pSeries:
hardware_interrupt_hv: hardware_interrupt_hv:
BEGIN_FTR_SECTION BEGIN_FTR_SECTION
_MASKABLE_EXCEPTION_PSERIES(0x500, hardware_interrupt,
EXC_STD, SOFTEN_TEST)
KVM_HANDLER(PACA_EXGEN, EXC_STD, 0x500)
FTR_SECTION_ELSE
_MASKABLE_EXCEPTION_PSERIES(0x502, hardware_interrupt, _MASKABLE_EXCEPTION_PSERIES(0x502, hardware_interrupt,
EXC_HV, SOFTEN_TEST_HV) EXC_HV, SOFTEN_TEST_HV)
KVM_HANDLER(PACA_EXGEN, EXC_HV, 0x502) KVM_HANDLER(PACA_EXGEN, EXC_HV, 0x502)
ALT_FTR_SECTION_END_IFCLR(CPU_FTR_HVMODE_206) FTR_SECTION_ELSE
_MASKABLE_EXCEPTION_PSERIES(0x500, hardware_interrupt,
EXC_STD, SOFTEN_TEST_PR)
KVM_HANDLER(PACA_EXGEN, EXC_STD, 0x500)
ALT_FTR_SECTION_END_IFSET(CPU_FTR_HVMODE_206)
STD_EXCEPTION_PSERIES(0x600, 0x600, alignment) STD_EXCEPTION_PSERIES(0x600, 0x600, alignment)
KVM_HANDLER(PACA_EXGEN, EXC_STD, 0x600) KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0x600)
STD_EXCEPTION_PSERIES(0x700, 0x700, program_check) STD_EXCEPTION_PSERIES(0x700, 0x700, program_check)
KVM_HANDLER(PACA_EXGEN, EXC_STD, 0x700) KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0x700)
STD_EXCEPTION_PSERIES(0x800, 0x800, fp_unavailable) STD_EXCEPTION_PSERIES(0x800, 0x800, fp_unavailable)
KVM_HANDLER(PACA_EXGEN, EXC_STD, 0x800) KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0x800)
MASKABLE_EXCEPTION_PSERIES(0x900, 0x900, decrementer) MASKABLE_EXCEPTION_PSERIES(0x900, 0x900, decrementer)
MASKABLE_EXCEPTION_HV(0x980, 0x982, decrementer) MASKABLE_EXCEPTION_HV(0x980, 0x982, decrementer)
STD_EXCEPTION_PSERIES(0xa00, 0xa00, trap_0a) STD_EXCEPTION_PSERIES(0xa00, 0xa00, trap_0a)
KVM_HANDLER(PACA_EXGEN, EXC_STD, 0xa00) KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xa00)
STD_EXCEPTION_PSERIES(0xb00, 0xb00, trap_0b) STD_EXCEPTION_PSERIES(0xb00, 0xb00, trap_0b)
KVM_HANDLER(PACA_EXGEN, EXC_STD, 0xb00) KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xb00)
. = 0xc00 . = 0xc00
.globl system_call_pSeries .globl system_call_pSeries
...@@ -219,7 +219,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_REAL_LE) ...@@ -219,7 +219,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_REAL_LE)
b . b .
STD_EXCEPTION_PSERIES(0xd00, 0xd00, single_step) STD_EXCEPTION_PSERIES(0xd00, 0xd00, single_step)
KVM_HANDLER(PACA_EXGEN, EXC_STD, 0xd00) KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xd00)
/* At 0xe??? we have a bunch of hypervisor exceptions, we branch /* At 0xe??? we have a bunch of hypervisor exceptions, we branch
* out of line to handle them * out of line to handle them
...@@ -254,23 +254,23 @@ vsx_unavailable_pSeries_1: ...@@ -254,23 +254,23 @@ vsx_unavailable_pSeries_1:
#ifdef CONFIG_CBE_RAS #ifdef CONFIG_CBE_RAS
STD_EXCEPTION_HV(0x1200, 0x1202, cbe_system_error) STD_EXCEPTION_HV(0x1200, 0x1202, cbe_system_error)
KVM_HANDLER_SKIP(PACA_EXGEN, EXC_HV, 0x1202) KVM_HANDLER_PR_SKIP(PACA_EXGEN, EXC_HV, 0x1202)
#endif /* CONFIG_CBE_RAS */ #endif /* CONFIG_CBE_RAS */
STD_EXCEPTION_PSERIES(0x1300, 0x1300, instruction_breakpoint) STD_EXCEPTION_PSERIES(0x1300, 0x1300, instruction_breakpoint)
KVM_HANDLER_SKIP(PACA_EXGEN, EXC_STD, 0x1300) KVM_HANDLER_PR_SKIP(PACA_EXGEN, EXC_STD, 0x1300)
#ifdef CONFIG_CBE_RAS #ifdef CONFIG_CBE_RAS
STD_EXCEPTION_HV(0x1600, 0x1602, cbe_maintenance) STD_EXCEPTION_HV(0x1600, 0x1602, cbe_maintenance)
KVM_HANDLER_SKIP(PACA_EXGEN, EXC_HV, 0x1602) KVM_HANDLER_PR_SKIP(PACA_EXGEN, EXC_HV, 0x1602)
#endif /* CONFIG_CBE_RAS */ #endif /* CONFIG_CBE_RAS */
STD_EXCEPTION_PSERIES(0x1700, 0x1700, altivec_assist) STD_EXCEPTION_PSERIES(0x1700, 0x1700, altivec_assist)
KVM_HANDLER(PACA_EXGEN, EXC_STD, 0x1700) KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0x1700)
#ifdef CONFIG_CBE_RAS #ifdef CONFIG_CBE_RAS
STD_EXCEPTION_HV(0x1800, 0x1802, cbe_thermal) STD_EXCEPTION_HV(0x1800, 0x1802, cbe_thermal)
KVM_HANDLER_SKIP(PACA_EXGEN, EXC_HV, 0x1802) KVM_HANDLER_PR_SKIP(PACA_EXGEN, EXC_HV, 0x1802)
#endif /* CONFIG_CBE_RAS */ #endif /* CONFIG_CBE_RAS */
. = 0x3000 . = 0x3000
...@@ -297,7 +297,7 @@ data_access_check_stab: ...@@ -297,7 +297,7 @@ data_access_check_stab:
mfspr r9,SPRN_DSISR mfspr r9,SPRN_DSISR
srdi r10,r10,60 srdi r10,r10,60
rlwimi r10,r9,16,0x20 rlwimi r10,r9,16,0x20
#ifdef CONFIG_KVM_BOOK3S_64_HANDLER #ifdef CONFIG_KVM_BOOK3S_PR
lbz r9,HSTATE_IN_GUEST(r13) lbz r9,HSTATE_IN_GUEST(r13)
rlwimi r10,r9,8,0x300 rlwimi r10,r9,8,0x300
#endif #endif
...@@ -316,11 +316,11 @@ do_stab_bolted_pSeries: ...@@ -316,11 +316,11 @@ do_stab_bolted_pSeries:
EXCEPTION_PROLOG_PSERIES_1(.do_stab_bolted, EXC_STD) EXCEPTION_PROLOG_PSERIES_1(.do_stab_bolted, EXC_STD)
#endif /* CONFIG_POWER4_ONLY */ #endif /* CONFIG_POWER4_ONLY */
KVM_HANDLER_SKIP(PACA_EXGEN, EXC_STD, 0x300) KVM_HANDLER_PR_SKIP(PACA_EXGEN, EXC_STD, 0x300)
KVM_HANDLER_SKIP(PACA_EXSLB, EXC_STD, 0x380) KVM_HANDLER_PR_SKIP(PACA_EXSLB, EXC_STD, 0x380)
KVM_HANDLER(PACA_EXGEN, EXC_STD, 0x400) KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0x400)
KVM_HANDLER(PACA_EXSLB, EXC_STD, 0x480) KVM_HANDLER_PR(PACA_EXSLB, EXC_STD, 0x480)
KVM_HANDLER(PACA_EXGEN, EXC_STD, 0x900) KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0x900)
KVM_HANDLER(PACA_EXGEN, EXC_HV, 0x982) KVM_HANDLER(PACA_EXGEN, EXC_HV, 0x982)
.align 7 .align 7
...@@ -336,11 +336,11 @@ do_stab_bolted_pSeries: ...@@ -336,11 +336,11 @@ do_stab_bolted_pSeries:
/* moved from 0xf00 */ /* moved from 0xf00 */
STD_EXCEPTION_PSERIES(., 0xf00, performance_monitor) STD_EXCEPTION_PSERIES(., 0xf00, performance_monitor)
KVM_HANDLER(PACA_EXGEN, EXC_STD, 0xf00) KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xf00)
STD_EXCEPTION_PSERIES(., 0xf20, altivec_unavailable) STD_EXCEPTION_PSERIES(., 0xf20, altivec_unavailable)
KVM_HANDLER(PACA_EXGEN, EXC_STD, 0xf20) KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xf20)
STD_EXCEPTION_PSERIES(., 0xf40, vsx_unavailable) STD_EXCEPTION_PSERIES(., 0xf40, vsx_unavailable)
KVM_HANDLER(PACA_EXGEN, EXC_STD, 0xf40) KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xf40)
/* /*
* An interrupt came in while soft-disabled; clear EE in SRR1, * An interrupt came in while soft-disabled; clear EE in SRR1,
...@@ -417,7 +417,11 @@ slb_miss_user_pseries: ...@@ -417,7 +417,11 @@ slb_miss_user_pseries:
/* KVM's trampoline code needs to be close to the interrupt handlers */ /* KVM's trampoline code needs to be close to the interrupt handlers */
#ifdef CONFIG_KVM_BOOK3S_64_HANDLER #ifdef CONFIG_KVM_BOOK3S_64_HANDLER
#ifdef CONFIG_KVM_BOOK3S_PR
#include "../kvm/book3s_rmhandlers.S" #include "../kvm/book3s_rmhandlers.S"
#else
#include "../kvm/book3s_hv_rmhandlers.S"
#endif
#endif #endif
.align 7 .align 7
......
...@@ -96,6 +96,7 @@ void flush_fp_to_thread(struct task_struct *tsk) ...@@ -96,6 +96,7 @@ void flush_fp_to_thread(struct task_struct *tsk)
preempt_enable(); preempt_enable();
} }
} }
EXPORT_SYMBOL_GPL(flush_fp_to_thread);
void enable_kernel_fp(void) void enable_kernel_fp(void)
{ {
...@@ -145,6 +146,7 @@ void flush_altivec_to_thread(struct task_struct *tsk) ...@@ -145,6 +146,7 @@ void flush_altivec_to_thread(struct task_struct *tsk)
preempt_enable(); preempt_enable();
} }
} }
EXPORT_SYMBOL_GPL(flush_altivec_to_thread);
#endif /* CONFIG_ALTIVEC */ #endif /* CONFIG_ALTIVEC */
#ifdef CONFIG_VSX #ifdef CONFIG_VSX
...@@ -186,6 +188,7 @@ void flush_vsx_to_thread(struct task_struct *tsk) ...@@ -186,6 +188,7 @@ void flush_vsx_to_thread(struct task_struct *tsk)
preempt_enable(); preempt_enable();
} }
} }
EXPORT_SYMBOL_GPL(flush_vsx_to_thread);
#endif /* CONFIG_VSX */ #endif /* CONFIG_VSX */
#ifdef CONFIG_SPE #ifdef CONFIG_SPE
......
...@@ -375,6 +375,9 @@ void __init check_for_initrd(void) ...@@ -375,6 +375,9 @@ void __init check_for_initrd(void)
int threads_per_core, threads_shift; int threads_per_core, threads_shift;
cpumask_t threads_core_mask; cpumask_t threads_core_mask;
EXPORT_SYMBOL_GPL(threads_per_core);
EXPORT_SYMBOL_GPL(threads_shift);
EXPORT_SYMBOL_GPL(threads_core_mask);
static void __init cpu_init_thread_core_maps(int tpc) static void __init cpu_init_thread_core_maps(int tpc)
{ {
......
...@@ -243,6 +243,7 @@ void smp_send_reschedule(int cpu) ...@@ -243,6 +243,7 @@ void smp_send_reschedule(int cpu)
if (likely(smp_ops)) if (likely(smp_ops))
smp_ops->message_pass(cpu, PPC_MSG_RESCHEDULE); smp_ops->message_pass(cpu, PPC_MSG_RESCHEDULE);
} }
EXPORT_SYMBOL_GPL(smp_send_reschedule);
void arch_send_call_function_single_ipi(int cpu) void arch_send_call_function_single_ipi(int cpu)
{ {
......
...@@ -20,7 +20,6 @@ config KVM ...@@ -20,7 +20,6 @@ config KVM
bool bool
select PREEMPT_NOTIFIERS select PREEMPT_NOTIFIERS
select ANON_INODES select ANON_INODES
select KVM_MMIO
config KVM_BOOK3S_HANDLER config KVM_BOOK3S_HANDLER
bool bool
...@@ -28,16 +27,22 @@ config KVM_BOOK3S_HANDLER ...@@ -28,16 +27,22 @@ config KVM_BOOK3S_HANDLER
config KVM_BOOK3S_32_HANDLER config KVM_BOOK3S_32_HANDLER
bool bool
select KVM_BOOK3S_HANDLER select KVM_BOOK3S_HANDLER
select KVM_MMIO
config KVM_BOOK3S_64_HANDLER config KVM_BOOK3S_64_HANDLER
bool bool
select KVM_BOOK3S_HANDLER select KVM_BOOK3S_HANDLER
config KVM_BOOK3S_PR
bool
select KVM_MMIO
config KVM_BOOK3S_32 config KVM_BOOK3S_32
tristate "KVM support for PowerPC book3s_32 processors" tristate "KVM support for PowerPC book3s_32 processors"
depends on EXPERIMENTAL && PPC_BOOK3S_32 && !SMP && !PTE_64BIT depends on EXPERIMENTAL && PPC_BOOK3S_32 && !SMP && !PTE_64BIT
select KVM select KVM
select KVM_BOOK3S_32_HANDLER select KVM_BOOK3S_32_HANDLER
select KVM_BOOK3S_PR
---help--- ---help---
Support running unmodified book3s_32 guest kernels Support running unmodified book3s_32 guest kernels
in virtual machines on book3s_32 host processors. in virtual machines on book3s_32 host processors.
...@@ -50,8 +55,8 @@ config KVM_BOOK3S_32 ...@@ -50,8 +55,8 @@ config KVM_BOOK3S_32
config KVM_BOOK3S_64 config KVM_BOOK3S_64
tristate "KVM support for PowerPC book3s_64 processors" tristate "KVM support for PowerPC book3s_64 processors"
depends on EXPERIMENTAL && PPC_BOOK3S_64 depends on EXPERIMENTAL && PPC_BOOK3S_64
select KVM
select KVM_BOOK3S_64_HANDLER select KVM_BOOK3S_64_HANDLER
select KVM
---help--- ---help---
Support running unmodified book3s_64 and book3s_32 guest kernels Support running unmodified book3s_64 and book3s_32 guest kernels
in virtual machines on book3s_64 host processors. in virtual machines on book3s_64 host processors.
...@@ -61,10 +66,37 @@ config KVM_BOOK3S_64 ...@@ -61,10 +66,37 @@ config KVM_BOOK3S_64
If unsure, say N. If unsure, say N.
config KVM_BOOK3S_64_HV
bool "KVM support for POWER7 using hypervisor mode in host"
depends on KVM_BOOK3S_64
---help---
Support running unmodified book3s_64 guest kernels in
virtual machines on POWER7 processors that have hypervisor
mode available to the host.
If you say Y here, KVM will use the hardware virtualization
facilities of POWER7 (and later) processors, meaning that
guest operating systems will run at full hardware speed
using supervisor and user modes. However, this also means
that KVM is not usable under PowerVM (pHyp), is only usable
on POWER7 (or later) processors, and can only emulate
POWER5+, POWER6 and POWER7 processors.
This module provides access to the hardware capabilities through
a character device node named /dev/kvm.
If unsure, say N.
config KVM_BOOK3S_64_PR
def_bool y
depends on KVM_BOOK3S_64 && !KVM_BOOK3S_64_HV
select KVM_BOOK3S_PR
config KVM_440 config KVM_440
bool "KVM support for PowerPC 440 processors" bool "KVM support for PowerPC 440 processors"
depends on EXPERIMENTAL && 44x depends on EXPERIMENTAL && 44x
select KVM select KVM
select KVM_MMIO
---help--- ---help---
Support running unmodified 440 guest kernels in virtual machines on Support running unmodified 440 guest kernels in virtual machines on
440 host processors. 440 host processors.
...@@ -89,6 +121,7 @@ config KVM_E500 ...@@ -89,6 +121,7 @@ config KVM_E500
bool "KVM support for PowerPC E500 processors" bool "KVM support for PowerPC E500 processors"
depends on EXPERIMENTAL && E500 depends on EXPERIMENTAL && E500
select KVM select KVM
select KVM_MMIO
---help--- ---help---
Support running unmodified E500 guest kernels in virtual machines on Support running unmodified E500 guest kernels in virtual machines on
E500 host processors. E500 host processors.
......
...@@ -38,11 +38,10 @@ kvm-e500-objs := \ ...@@ -38,11 +38,10 @@ kvm-e500-objs := \
e500_emulate.o e500_emulate.o
kvm-objs-$(CONFIG_KVM_E500) := $(kvm-e500-objs) kvm-objs-$(CONFIG_KVM_E500) := $(kvm-e500-objs)
kvm-book3s_64-objs := \ kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_PR) := \
$(common-objs-y) \ ../../../virt/kvm/coalesced_mmio.o \
fpu.o \ fpu.o \
book3s_paired_singles.o \ book3s_paired_singles.o \
book3s.o \
book3s_pr.o \ book3s_pr.o \
book3s_emulate.o \ book3s_emulate.o \
book3s_interrupts.o \ book3s_interrupts.o \
...@@ -50,6 +49,18 @@ kvm-book3s_64-objs := \ ...@@ -50,6 +49,18 @@ kvm-book3s_64-objs := \
book3s_64_mmu_host.o \ book3s_64_mmu_host.o \
book3s_64_mmu.o \ book3s_64_mmu.o \
book3s_32_mmu.o book3s_32_mmu.o
kvm-book3s_64-objs-$(CONFIG_KVM_BOOK3S_64_HV) := \
book3s_hv.o \
book3s_hv_interrupts.o \
book3s_64_mmu_hv.o
kvm-book3s_64-objs := \
../../../virt/kvm/kvm_main.o \
powerpc.o \
emulate.o \
book3s.o \
$(kvm-book3s_64-objs-y)
kvm-objs-$(CONFIG_KVM_BOOK3S_64) := $(kvm-book3s_64-objs) kvm-objs-$(CONFIG_KVM_BOOK3S_64) := $(kvm-book3s_64-objs)
kvm-book3s_32-objs := \ kvm-book3s_32-objs := \
......
/*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License, version 2, as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* Copyright 2010 Paul Mackerras, IBM Corp. <paulus@au1.ibm.com>
*/
#include <linux/types.h>
#include <linux/string.h>
#include <linux/kvm.h>
#include <linux/kvm_host.h>
#include <linux/highmem.h>
#include <linux/gfp.h>
#include <linux/slab.h>
#include <linux/hugetlb.h>
#include <asm/tlbflush.h>
#include <asm/kvm_ppc.h>
#include <asm/kvm_book3s.h>
#include <asm/mmu-hash64.h>
#include <asm/hvcall.h>
#include <asm/synch.h>
#include <asm/ppc-opcode.h>
#include <asm/cputable.h>
/* For now use fixed-size 16MB page table */
#define HPT_ORDER 24
#define HPT_NPTEG (1ul << (HPT_ORDER - 7)) /* 128B per pteg */
#define HPT_HASH_MASK (HPT_NPTEG - 1)
/* Pages in the VRMA are 16MB pages */
#define VRMA_PAGE_ORDER 24
#define VRMA_VSID 0x1ffffffUL /* 1TB VSID reserved for VRMA */
#define NR_LPIDS (LPID_RSVD + 1)
unsigned long lpid_inuse[BITS_TO_LONGS(NR_LPIDS)];
long kvmppc_alloc_hpt(struct kvm *kvm)
{
unsigned long hpt;
unsigned long lpid;
hpt = __get_free_pages(GFP_KERNEL|__GFP_ZERO|__GFP_REPEAT|__GFP_NOWARN,
HPT_ORDER - PAGE_SHIFT);
if (!hpt) {
pr_err("kvm_alloc_hpt: Couldn't alloc HPT\n");
return -ENOMEM;
}
kvm->arch.hpt_virt = hpt;
do {
lpid = find_first_zero_bit(lpid_inuse, NR_LPIDS);
if (lpid >= NR_LPIDS) {
pr_err("kvm_alloc_hpt: No LPIDs free\n");
free_pages(hpt, HPT_ORDER - PAGE_SHIFT);
return -ENOMEM;
}
} while (test_and_set_bit(lpid, lpid_inuse));
kvm->arch.sdr1 = __pa(hpt) | (HPT_ORDER - 18);
kvm->arch.lpid = lpid;
kvm->arch.host_sdr1 = mfspr(SPRN_SDR1);
kvm->arch.host_lpid = mfspr(SPRN_LPID);
kvm->arch.host_lpcr = mfspr(SPRN_LPCR);
pr_info("KVM guest htab at %lx, LPID %lx\n", hpt, lpid);
return 0;
}
void kvmppc_free_hpt(struct kvm *kvm)
{
unsigned long i;
struct kvmppc_pginfo *pginfo;
clear_bit(kvm->arch.lpid, lpid_inuse);
free_pages(kvm->arch.hpt_virt, HPT_ORDER - PAGE_SHIFT);
if (kvm->arch.ram_pginfo) {
pginfo = kvm->arch.ram_pginfo;
kvm->arch.ram_pginfo = NULL;
for (i = 0; i < kvm->arch.ram_npages; ++i)
put_page(pfn_to_page(pginfo[i].pfn));
kfree(pginfo);
}
}
static unsigned long user_page_size(unsigned long addr)
{
struct vm_area_struct *vma;
unsigned long size = PAGE_SIZE;
down_read(&current->mm->mmap_sem);
vma = find_vma(current->mm, addr);
if (vma)
size = vma_kernel_pagesize(vma);
up_read(&current->mm->mmap_sem);
return size;
}
static pfn_t hva_to_pfn(unsigned long addr)
{
struct page *page[1];
int npages;
might_sleep();
npages = get_user_pages_fast(addr, 1, 1, page);
if (unlikely(npages != 1))
return 0;
return page_to_pfn(page[0]);
}
long kvmppc_prepare_vrma(struct kvm *kvm,
struct kvm_userspace_memory_region *mem)
{
unsigned long psize, porder;
unsigned long i, npages;
struct kvmppc_pginfo *pginfo;
pfn_t pfn;
unsigned long hva;
/* First see what page size we have */
psize = user_page_size(mem->userspace_addr);
/* For now, only allow 16MB pages */
if (psize != 1ul << VRMA_PAGE_ORDER || (mem->memory_size & (psize - 1))) {
pr_err("bad psize=%lx memory_size=%llx @ %llx\n",
psize, mem->memory_size, mem->userspace_addr);
return -EINVAL;
}
porder = __ilog2(psize);
npages = mem->memory_size >> porder;
pginfo = kzalloc(npages * sizeof(struct kvmppc_pginfo), GFP_KERNEL);
if (!pginfo) {
pr_err("kvmppc_prepare_vrma: couldn't alloc %lu bytes\n",
npages * sizeof(struct kvmppc_pginfo));
return -ENOMEM;
}
for (i = 0; i < npages; ++i) {
hva = mem->userspace_addr + (i << porder);
if (user_page_size(hva) != psize)
goto err;
pfn = hva_to_pfn(hva);
if (pfn == 0) {
pr_err("oops, no pfn for hva %lx\n", hva);
goto err;
}
if (pfn & ((1ul << (porder - PAGE_SHIFT)) - 1)) {
pr_err("oops, unaligned pfn %llx\n", pfn);
put_page(pfn_to_page(pfn));
goto err;
}
pginfo[i].pfn = pfn;
}
kvm->arch.ram_npages = npages;
kvm->arch.ram_psize = psize;
kvm->arch.ram_porder = porder;
kvm->arch.ram_pginfo = pginfo;
return 0;
err:
kfree(pginfo);
return -EINVAL;
}
void kvmppc_map_vrma(struct kvm *kvm, struct kvm_userspace_memory_region *mem)
{
unsigned long i;
unsigned long npages = kvm->arch.ram_npages;
unsigned long pfn;
unsigned long *hpte;
unsigned long hash;
struct kvmppc_pginfo *pginfo = kvm->arch.ram_pginfo;
if (!pginfo)
return;
/* VRMA can't be > 1TB */
if (npages > 1ul << (40 - kvm->arch.ram_porder))
npages = 1ul << (40 - kvm->arch.ram_porder);
/* Can't use more than 1 HPTE per HPTEG */
if (npages > HPT_NPTEG)
npages = HPT_NPTEG;
for (i = 0; i < npages; ++i) {
pfn = pginfo[i].pfn;
/* can't use hpt_hash since va > 64 bits */
hash = (i ^ (VRMA_VSID ^ (VRMA_VSID << 25))) & HPT_HASH_MASK;
/*
* We assume that the hash table is empty and no
* vcpus are using it at this stage. Since we create
* at most one HPTE per HPTEG, we just assume entry 7
* is available and use it.
*/
hpte = (unsigned long *) (kvm->arch.hpt_virt + (hash << 7));
hpte += 7 * 2;
/* HPTE low word - RPN, protection, etc. */
hpte[1] = (pfn << PAGE_SHIFT) | HPTE_R_R | HPTE_R_C |
HPTE_R_M | PP_RWXX;
wmb();
hpte[0] = HPTE_V_1TB_SEG | (VRMA_VSID << (40 - 16)) |
(i << (VRMA_PAGE_ORDER - 16)) | HPTE_V_BOLTED |
HPTE_V_LARGE | HPTE_V_VALID;
}
}
int kvmppc_mmu_hv_init(void)
{
if (!cpu_has_feature(CPU_FTR_HVMODE_206))
return -EINVAL;
memset(lpid_inuse, 0, sizeof(lpid_inuse));
set_bit(mfspr(SPRN_LPID), lpid_inuse);
set_bit(LPID_RSVD, lpid_inuse);
return 0;
}
void kvmppc_mmu_destroy(struct kvm_vcpu *vcpu)
{
}
static void kvmppc_mmu_book3s_64_hv_reset_msr(struct kvm_vcpu *vcpu)
{
kvmppc_set_msr(vcpu, MSR_SF | MSR_ME);
}
static int kvmppc_mmu_book3s_64_hv_xlate(struct kvm_vcpu *vcpu, gva_t eaddr,
struct kvmppc_pte *gpte, bool data)
{
return -ENOENT;
}
void kvmppc_mmu_book3s_hv_init(struct kvm_vcpu *vcpu)
{
struct kvmppc_mmu *mmu = &vcpu->arch.mmu;
vcpu->arch.slb_nr = 32; /* Assume POWER7 for now */
mmu->xlate = kvmppc_mmu_book3s_64_hv_xlate;
mmu->reset_msr = kvmppc_mmu_book3s_64_hv_reset_msr;
vcpu->arch.hflags |= BOOK3S_HFLAG_SLB;
}
...@@ -20,6 +20,9 @@ ...@@ -20,6 +20,9 @@
#include <linux/module.h> #include <linux/module.h>
#include <asm/kvm_book3s.h> #include <asm/kvm_book3s.h>
#ifdef CONFIG_KVM_BOOK3S_64_HV
EXPORT_SYMBOL_GPL(kvmppc_hv_entry_trampoline);
#else
EXPORT_SYMBOL_GPL(kvmppc_handler_trampoline_enter); EXPORT_SYMBOL_GPL(kvmppc_handler_trampoline_enter);
EXPORT_SYMBOL_GPL(kvmppc_handler_lowmem_trampoline); EXPORT_SYMBOL_GPL(kvmppc_handler_lowmem_trampoline);
EXPORT_SYMBOL_GPL(kvmppc_rmcall); EXPORT_SYMBOL_GPL(kvmppc_rmcall);
...@@ -30,3 +33,5 @@ EXPORT_SYMBOL_GPL(kvmppc_load_up_altivec); ...@@ -30,3 +33,5 @@ EXPORT_SYMBOL_GPL(kvmppc_load_up_altivec);
#ifdef CONFIG_VSX #ifdef CONFIG_VSX
EXPORT_SYMBOL_GPL(kvmppc_load_up_vsx); EXPORT_SYMBOL_GPL(kvmppc_load_up_vsx);
#endif #endif
#endif
This diff is collapsed.
/*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License, version 2, as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* Copyright 2011 Paul Mackerras, IBM Corp. <paulus@au1.ibm.com>
*
* Derived from book3s_interrupts.S, which is:
* Copyright SUSE Linux Products GmbH 2009
*
* Authors: Alexander Graf <agraf@suse.de>
*/
#include <asm/ppc_asm.h>
#include <asm/kvm_asm.h>
#include <asm/reg.h>
#include <asm/page.h>
#include <asm/asm-offsets.h>
#include <asm/exception-64s.h>
#include <asm/ppc-opcode.h>
/*****************************************************************************
* *
* Guest entry / exit code that is in kernel module memory (vmalloc) *
* *
****************************************************************************/
/* Registers:
* r4: vcpu pointer
*/
_GLOBAL(__kvmppc_vcore_entry)
/* Write correct stack frame */
mflr r0
std r0,PPC_LR_STKOFF(r1)
/* Save host state to the stack */
stdu r1, -SWITCH_FRAME_SIZE(r1)
/* Save non-volatile registers (r14 - r31) */
SAVE_NVGPRS(r1)
/* Save host DSCR */
mfspr r3, SPRN_DSCR
std r3, HSTATE_DSCR(r13)
/* Save host DABR */
mfspr r3, SPRN_DABR
std r3, HSTATE_DABR(r13)
/* Hard-disable interrupts */
mfmsr r10
std r10, HSTATE_HOST_MSR(r13)
rldicl r10,r10,48,1
rotldi r10,r10,16
mtmsrd r10,1
/* Save host PMU registers and load guest PMU registers */
/* R4 is live here (vcpu pointer) but not r3 or r5 */
li r3, 1
sldi r3, r3, 31 /* MMCR0_FC (freeze counters) bit */
mfspr r7, SPRN_MMCR0 /* save MMCR0 */
mtspr SPRN_MMCR0, r3 /* freeze all counters, disable interrupts */
isync
ld r3, PACALPPACAPTR(r13) /* is the host using the PMU? */
lbz r5, LPPACA_PMCINUSE(r3)
cmpwi r5, 0
beq 31f /* skip if not */
mfspr r5, SPRN_MMCR1
mfspr r6, SPRN_MMCRA
std r7, HSTATE_MMCR(r13)
std r5, HSTATE_MMCR + 8(r13)
std r6, HSTATE_MMCR + 16(r13)
mfspr r3, SPRN_PMC1
mfspr r5, SPRN_PMC2
mfspr r6, SPRN_PMC3
mfspr r7, SPRN_PMC4
mfspr r8, SPRN_PMC5
mfspr r9, SPRN_PMC6
stw r3, HSTATE_PMC(r13)
stw r5, HSTATE_PMC + 4(r13)
stw r6, HSTATE_PMC + 8(r13)
stw r7, HSTATE_PMC + 12(r13)
stw r8, HSTATE_PMC + 16(r13)
stw r9, HSTATE_PMC + 20(r13)
31:
/*
* Put whatever is in the decrementer into the
* hypervisor decrementer.
*/
mfspr r8,SPRN_DEC
mftb r7
mtspr SPRN_HDEC,r8
extsw r8,r8
add r8,r8,r7
std r8,HSTATE_DECEXP(r13)
/* Jump to partition switch code */
bl .kvmppc_hv_entry_trampoline
nop
/*
* We return here in virtual mode after the guest exits
* with something that we can't handle in real mode.
* Interrupts are enabled again at this point.
*/
.global kvmppc_handler_highmem
kvmppc_handler_highmem:
/*
* Register usage at this point:
*
* R1 = host R1
* R2 = host R2
* R12 = exit handler id
* R13 = PACA
*/
/* Restore non-volatile host registers (r14 - r31) */
REST_NVGPRS(r1)
addi r1, r1, SWITCH_FRAME_SIZE
ld r0, PPC_LR_STKOFF(r1)
mtlr r0
blr
This diff is collapsed.
...@@ -89,29 +89,29 @@ kvmppc_handler_trampoline_enter: ...@@ -89,29 +89,29 @@ kvmppc_handler_trampoline_enter:
/* Enter guest */ /* Enter guest */
PPC_LL r4, (SVCPU_CTR)(r3) PPC_LL r4, SVCPU_CTR(r3)
PPC_LL r5, (SVCPU_LR)(r3) PPC_LL r5, SVCPU_LR(r3)
lwz r6, (SVCPU_CR)(r3) lwz r6, SVCPU_CR(r3)
lwz r7, (SVCPU_XER)(r3) lwz r7, SVCPU_XER(r3)
mtctr r4 mtctr r4
mtlr r5 mtlr r5
mtcr r6 mtcr r6
mtxer r7 mtxer r7
PPC_LL r0, (SVCPU_R0)(r3) PPC_LL r0, SVCPU_R0(r3)
PPC_LL r1, (SVCPU_R1)(r3) PPC_LL r1, SVCPU_R1(r3)
PPC_LL r2, (SVCPU_R2)(r3) PPC_LL r2, SVCPU_R2(r3)
PPC_LL r4, (SVCPU_R4)(r3) PPC_LL r4, SVCPU_R4(r3)
PPC_LL r5, (SVCPU_R5)(r3) PPC_LL r5, SVCPU_R5(r3)
PPC_LL r6, (SVCPU_R6)(r3) PPC_LL r6, SVCPU_R6(r3)
PPC_LL r7, (SVCPU_R7)(r3) PPC_LL r7, SVCPU_R7(r3)
PPC_LL r8, (SVCPU_R8)(r3) PPC_LL r8, SVCPU_R8(r3)
PPC_LL r9, (SVCPU_R9)(r3) PPC_LL r9, SVCPU_R9(r3)
PPC_LL r10, (SVCPU_R10)(r3) PPC_LL r10, SVCPU_R10(r3)
PPC_LL r11, (SVCPU_R11)(r3) PPC_LL r11, SVCPU_R11(r3)
PPC_LL r12, (SVCPU_R12)(r3) PPC_LL r12, SVCPU_R12(r3)
PPC_LL r13, (SVCPU_R13)(r3) PPC_LL r13, SVCPU_R13(r3)
PPC_LL r3, (SVCPU_R3)(r3) PPC_LL r3, (SVCPU_R3)(r3)
......
...@@ -38,8 +38,12 @@ ...@@ -38,8 +38,12 @@
int kvm_arch_vcpu_runnable(struct kvm_vcpu *v) int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
{ {
#ifndef CONFIG_KVM_BOOK3S_64_HV
return !(v->arch.shared->msr & MSR_WE) || return !(v->arch.shared->msr & MSR_WE) ||
!!(v->arch.pending_exceptions); !!(v->arch.pending_exceptions);
#else
return 1;
#endif
} }
int kvmppc_kvm_pv(struct kvm_vcpu *vcpu) int kvmppc_kvm_pv(struct kvm_vcpu *vcpu)
...@@ -184,10 +188,13 @@ int kvm_dev_ioctl_check_extension(long ext) ...@@ -184,10 +188,13 @@ int kvm_dev_ioctl_check_extension(long ext)
#else #else
case KVM_CAP_PPC_SEGSTATE: case KVM_CAP_PPC_SEGSTATE:
#endif #endif
case KVM_CAP_PPC_PAIRED_SINGLES:
case KVM_CAP_PPC_UNSET_IRQ: case KVM_CAP_PPC_UNSET_IRQ:
case KVM_CAP_PPC_IRQ_LEVEL: case KVM_CAP_PPC_IRQ_LEVEL:
case KVM_CAP_ENABLE_CAP: case KVM_CAP_ENABLE_CAP:
r = 1;
break;
#ifndef CONFIG_KVM_BOOK3S_64_HV
case KVM_CAP_PPC_PAIRED_SINGLES:
case KVM_CAP_PPC_OSI: case KVM_CAP_PPC_OSI:
case KVM_CAP_PPC_GET_PVINFO: case KVM_CAP_PPC_GET_PVINFO:
r = 1; r = 1;
...@@ -195,6 +202,7 @@ int kvm_dev_ioctl_check_extension(long ext) ...@@ -195,6 +202,7 @@ int kvm_dev_ioctl_check_extension(long ext)
case KVM_CAP_COALESCED_MMIO: case KVM_CAP_COALESCED_MMIO:
r = KVM_COALESCED_MMIO_PAGE_OFFSET; r = KVM_COALESCED_MMIO_PAGE_OFFSET;
break; break;
#endif
default: default:
r = 0; r = 0;
break; break;
...@@ -291,6 +299,7 @@ int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu) ...@@ -291,6 +299,7 @@ int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu)
hrtimer_init(&vcpu->arch.dec_timer, CLOCK_REALTIME, HRTIMER_MODE_ABS); hrtimer_init(&vcpu->arch.dec_timer, CLOCK_REALTIME, HRTIMER_MODE_ABS);
tasklet_init(&vcpu->arch.tasklet, kvmppc_decrementer_func, (ulong)vcpu); tasklet_init(&vcpu->arch.tasklet, kvmppc_decrementer_func, (ulong)vcpu);
vcpu->arch.dec_timer.function = kvmppc_decrementer_wakeup; vcpu->arch.dec_timer.function = kvmppc_decrementer_wakeup;
vcpu->arch.dec_expires = ~(u64)0;
#ifdef CONFIG_KVM_EXIT_TIMING #ifdef CONFIG_KVM_EXIT_TIMING
mutex_init(&vcpu->arch.exit_timing_lock); mutex_init(&vcpu->arch.exit_timing_lock);
...@@ -317,6 +326,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) ...@@ -317,6 +326,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
mtspr(SPRN_VRSAVE, vcpu->arch.vrsave); mtspr(SPRN_VRSAVE, vcpu->arch.vrsave);
#endif #endif
kvmppc_core_vcpu_load(vcpu, cpu); kvmppc_core_vcpu_load(vcpu, cpu);
vcpu->cpu = smp_processor_id();
} }
void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
...@@ -325,6 +335,7 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) ...@@ -325,6 +335,7 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
#ifdef CONFIG_BOOKE #ifdef CONFIG_BOOKE
vcpu->arch.vrsave = mfspr(SPRN_VRSAVE); vcpu->arch.vrsave = mfspr(SPRN_VRSAVE);
#endif #endif
vcpu->cpu = -1;
} }
int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu, int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,
...@@ -496,6 +507,13 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run) ...@@ -496,6 +507,13 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
for (i = 0; i < 32; i++) for (i = 0; i < 32; i++)
kvmppc_set_gpr(vcpu, i, gprs[i]); kvmppc_set_gpr(vcpu, i, gprs[i]);
vcpu->arch.osi_needed = 0; vcpu->arch.osi_needed = 0;
} else if (vcpu->arch.hcall_needed) {
int i;
kvmppc_set_gpr(vcpu, 3, run->papr_hcall.ret);
for (i = 0; i < 9; ++i)
kvmppc_set_gpr(vcpu, 4 + i, run->papr_hcall.args[i]);
vcpu->arch.hcall_needed = 0;
} }
kvmppc_core_deliver_interrupts(vcpu); kvmppc_core_deliver_interrupts(vcpu);
...@@ -518,6 +536,8 @@ int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu, struct kvm_interrupt *irq) ...@@ -518,6 +536,8 @@ int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu, struct kvm_interrupt *irq)
if (waitqueue_active(&vcpu->wq)) { if (waitqueue_active(&vcpu->wq)) {
wake_up_interruptible(&vcpu->wq); wake_up_interruptible(&vcpu->wq);
vcpu->stat.halt_wakeup++; vcpu->stat.halt_wakeup++;
} else if (vcpu->cpu != -1) {
smp_send_reschedule(vcpu->cpu);
} }
return 0; return 0;
......
...@@ -103,7 +103,7 @@ TRACE_EVENT(kvm_gtlb_write, ...@@ -103,7 +103,7 @@ TRACE_EVENT(kvm_gtlb_write,
* Book3S trace points * * Book3S trace points *
*************************************************************************/ *************************************************************************/
#ifdef CONFIG_PPC_BOOK3S #ifdef CONFIG_KVM_BOOK3S_PR
TRACE_EVENT(kvm_book3s_exit, TRACE_EVENT(kvm_book3s_exit,
TP_PROTO(unsigned int exit_nr, struct kvm_vcpu *vcpu), TP_PROTO(unsigned int exit_nr, struct kvm_vcpu *vcpu),
......
...@@ -161,6 +161,7 @@ struct kvm_pit_config { ...@@ -161,6 +161,7 @@ struct kvm_pit_config {
#define KVM_EXIT_NMI 16 #define KVM_EXIT_NMI 16
#define KVM_EXIT_INTERNAL_ERROR 17 #define KVM_EXIT_INTERNAL_ERROR 17
#define KVM_EXIT_OSI 18 #define KVM_EXIT_OSI 18
#define KVM_EXIT_PAPR_HCALL 19
/* For KVM_EXIT_INTERNAL_ERROR */ /* For KVM_EXIT_INTERNAL_ERROR */
#define KVM_INTERNAL_ERROR_EMULATION 1 #define KVM_INTERNAL_ERROR_EMULATION 1
...@@ -264,6 +265,11 @@ struct kvm_run { ...@@ -264,6 +265,11 @@ struct kvm_run {
struct { struct {
__u64 gprs[32]; __u64 gprs[32];
} osi; } osi;
struct {
__u64 nr;
__u64 ret;
__u64 args[9];
} papr_hcall;
/* Fix the size of the union. */ /* Fix the size of the union. */
char padding[256]; char padding[256];
}; };
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment