Skip to content

L
linux
Commit 1765a1fe authored by Linus Torvalds's avatar Linus Torvalds
Browse files
Options

Merge branch 'kvm-updates/2.6.37' of git://git.kernel.org/pub/scm/virt/kvm/kvm 

* 'kvm-updates/2.6.37' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (321 commits)
  KVM: Drop CONFIG_DMAR dependency around kvm_iommu_map_pages
  KVM: Fix signature of kvm_iommu_map_pages stub
  KVM: MCE: Send SRAR SIGBUS directly
  KVM: MCE: Add MCG_SER_P into KVM_MCE_CAP_SUPPORTED
  KVM: fix typo in copyright notice
  KVM: Disable interrupts around get_kernel_ns()
  KVM: MMU: Avoid sign extension in mmu_alloc_direct_roots() pae root address
  KVM: MMU: move access code parsing to FNAME(walk_addr) function
  KVM: MMU: audit: check whether have unsync sps after root sync
  KVM: MMU: audit: introduce audit_printk to cleanup audit code
  KVM: MMU: audit: unregister audit tracepoints before module unloaded
  KVM: MMU: audit: fix vcpu's spte walking
  KVM: MMU: set access bit for direct mapping
  KVM: MMU: cleanup for error mask set while walk guest page table
  KVM: MMU: update 'root_hpa' out of loop in PAE shadow path
  KVM: x86 emulator: Eliminate compilation warning in x86_decode_insn()
  KVM: x86: Fix constant type in kvm_get_time_scale
  KVM: VMX: Add AX to list of registers clobbered by guest switch
  KVM guest: Move a printk that's using the clock before it's ready
  KVM: x86: TSC catchup mode
  ...
parents bdaf12b4 2a31339a
Expand all Show whitespace changes
Inline Side-by-side
Showing with 6701 additions and 2389 deletions
Documentation/kernel-parameters.txt
View file @ 1765a1fe
......@@ -1131,9 +1131,13 @@ and is between 256 and 4096 characters. It is defined in the file
kvm.oos_shadow= [KVM] Disable out-of-sync shadow paging.
Default is 1 (enabled)
kvm-amd.nested= [KVM,AMD] Allow nested virtualization in KVM/SVM.
kvm.mmu_audit= [KVM] This is a R/W parameter which allows audit
KVM MMU at runtime.
Default is 0 (off)
kvm-amd.nested= [KVM,AMD] Allow nested virtualization in KVM/SVM.
Default is 1 (enabled)
kvm-amd.npt= [KVM,AMD] Disable nested paging (virtualized MMU)
for all guests.
Default is 1 (enabled) if in 64bit or 32bit-PAE mode
......@@ -1698,6 +1702,8 @@ and is between 256 and 4096 characters. It is defined in the file
nojitter [IA64] Disables jitter checking for ITC timers.
no-kvmclock [X86,KVM] Disable paravirtualized KVM clock driver
nolapic [X86-32,APIC] Do not enable or use the local APIC.
nolapic_timer [X86-32,APIC] Do not use the local APIC timer.
......
Documentation/kvm/api.txt
View file @ 1765a1fe
......@@ -320,13 +320,13 @@ struct kvm_translation {
4.15 KVM_INTERRUPT
Capability: basic
Architectures: x86
Architectures: x86, ppc
Type: vcpu ioctl
Parameters: struct kvm_interrupt (in)
Returns: 0 on success, -1 on error
Queues a hardware interrupt vector to be injected. This is only
useful if in-kernel local APIC is not used.
useful if in-kernel local APIC or equivalent is not used.
/* for KVM_INTERRUPT */
struct kvm_interrupt {
......@@ -334,8 +334,37 @@ struct kvm_interrupt {
__u32 irq;
};
X86:
Note 'irq' is an interrupt vector, not an interrupt pin or line.
PPC:
Queues an external interrupt to be injected. This ioctl is overleaded
with 3 different irq values:
a) KVM_INTERRUPT_SET
This injects an edge type external interrupt into the guest once it's ready
to receive interrupts. When injected, the interrupt is done.
b) KVM_INTERRUPT_UNSET
This unsets any pending interrupt.
Only available with KVM_CAP_PPC_UNSET_IRQ.
c) KVM_INTERRUPT_SET_LEVEL
This injects a level type external interrupt into the guest context. The
interrupt stays pending until a specific ioctl with KVM_INTERRUPT_UNSET
is triggered.
Only available with KVM_CAP_PPC_IRQ_LEVEL.
Note that any value for 'irq' other than the ones stated above is invalid
and incurs unexpected behavior.
4.16 KVM_DEBUG_GUEST
Capability: basic
......@@ -1013,8 +1042,9 @@ number is just right, the 'nent' field is adjusted to the number of valid
entries in the 'entries' array, which is then filled.
The entries returned are the host cpuid as returned by the cpuid instruction,
with unknown or unsupported features masked out. The fields in each entry
are defined as follows:
with unknown or unsupported features masked out. Some features (for example,
x2apic), may not be present in the host cpu, but are exposed by kvm if it can
emulate them efficiently. The fields in each entry are defined as follows:
function: the eax value used to obtain the entry
index: the ecx value used to obtain the entry (for entries that are
......@@ -1032,6 +1062,29 @@ are defined as follows:
eax, ebx, ecx, edx: the values returned by the cpuid instruction for
this function/index combination
4.46 KVM_PPC_GET_PVINFO
Capability: KVM_CAP_PPC_GET_PVINFO
Architectures: ppc
Type: vm ioctl
Parameters: struct kvm_ppc_pvinfo (out)
Returns: 0 on success, !0 on error
struct kvm_ppc_pvinfo {
__u32 flags;
__u32 hcall[4];
__u8 pad[108];
};
This ioctl fetches PV specific information that need to be passed to the guest
using the device tree or other means from vm context.
For now the only implemented piece of information distributed here is an array
of 4 instructions that make up a hypercall.
If any additional field gets added to this structure later on, a bit for that
additional piece of information will be set in the flags bitmap.
5. The kvm_run structure
Application code obtains a pointer to the kvm_run structure by
......
Documentation/kvm/ppc-pv.txt 0 → 100644
View file @ 1765a1fe
The PPC KVM paravirtual interface
=================================
The basic execution principle by which KVM on PowerPC works is to run all kernel
space code in PR=1 which is user space. This way we trap all privileged
instructions and can emulate them accordingly.
Unfortunately that is also the downfall. There are quite some privileged
instructions that needlessly return us to the hypervisor even though they
could be handled differently.
This is what the PPC PV interface helps with. It takes privileged instructions
and transforms them into unprivileged ones with some help from the hypervisor.
This cuts down virtualization costs by about 50% on some of my benchmarks.
The code for that interface can be found in arch/powerpc/kernel/kvm*
Querying for existence
======================
To find out if we're running on KVM or not, we leverage the device tree. When
Linux is running on KVM, a node /hypervisor exists. That node contains a
compatible property with the value "linux,kvm".
Once you determined you're running under a PV capable KVM, you can now use
hypercalls as described below.
KVM hypercalls
==============
Inside the device tree's /hypervisor node there's a property called
'hypercall-instructions'. This property contains at most 4 opcodes that make
up the hypercall. To call a hypercall, just call these instructions.
The parameters are as follows:
Register IN OUT
r0 - volatile
r3 1st parameter Return code
r4 2nd parameter 1st output value
r5 3rd parameter 2nd output value
r6 4th parameter 3rd output value
r7 5th parameter 4th output value
r8 6th parameter 5th output value
r9 7th parameter 6th output value
r10 8th parameter 7th output value
r11 hypercall number 8th output value
r12 - volatile
Hypercall definitions are shared in generic code, so the same hypercall numbers
apply for x86 and powerpc alike with the exception that each KVM hypercall
also needs to be ORed with the KVM vendor code which is (42 << 16).
Return codes can be as follows:
Code Meaning
0 Success
12 Hypercall not implemented
<0 Error
The magic page
==============
To enable communication between the hypervisor and guest there is a new shared
page that contains parts of supervisor visible register state. The guest can
map this shared page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE.
With this hypercall issued the guest always gets the magic page mapped at the
desired location in effective and physical address space. For now, we always
map the page to -4096. This way we can access it using absolute load and store
functions. The following instruction reads the first field of the magic page:
ld rX, -4096(0)
The interface is designed to be extensible should there be need later to add
additional registers to the magic page. If you add fields to the magic page,
also define a new hypercall feature to indicate that the host can give you more
registers. Only if the host supports the additional features, make use of them.
The magic page has the following layout as described in
arch/powerpc/include/asm/kvm_para.h:
struct kvm_vcpu_arch_shared {
__u64 scratch1;
__u64 scratch2;
__u64 scratch3;
__u64 critical; /* Guest may not get interrupts if == r1 */
__u64 sprg0;
__u64 sprg1;
__u64 sprg2;
__u64 sprg3;
__u64 srr0;
__u64 srr1;
__u64 dar;
__u64 msr;
__u32 dsisr;
__u32 int_pending; /* Tells the guest if we have an interrupt */
};
Additions to the page must only occur at the end. Struct fields are always 32
or 64 bit aligned, depending on them being 32 or 64 bit wide respectively.
Magic page features
===================
When mapping the magic page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE,
a second return value is passed to the guest. This second return value contains
a bitmap of available features inside the magic page.
The following enhancements to the magic page are currently available:
KVM_MAGIC_FEAT_SR Maps SR registers r/w in the magic page
For enhanced features in the magic page, please check for the existence of the
feature before using them!
MSR bits
========
The MSR contains bits that require hypervisor intervention and bits that do
not require direct hypervisor intervention because they only get interpreted
when entering the guest or don't have any impact on the hypervisor's behavior.
The following bits are safe to be set inside the guest:
MSR_EE
MSR_RI
MSR_CR
MSR_ME
If any other bit changes in the MSR, please still use mtmsr(d).
Patched instructions
====================
The "ld" and "std" instructions are transormed to "lwz" and "stw" instructions
respectively on 32 bit systems with an added offset of 4 to accomodate for big
endianness.
The following is a list of mapping the Linux kernel performs when running as
guest. Implementing any of those mappings is optional, as the instruction traps
also act on the shared page. So calling privileged instructions still works as
before.
From To
==== ==
mfmsr rX ld rX, magic_page->msr
mfsprg rX, 0 ld rX, magic_page->sprg0
mfsprg rX, 1 ld rX, magic_page->sprg1
mfsprg rX, 2 ld rX, magic_page->sprg2
mfsprg rX, 3 ld rX, magic_page->sprg3
mfsrr0 rX ld rX, magic_page->srr0
mfsrr1 rX ld rX, magic_page->srr1
mfdar rX ld rX, magic_page->dar
mfdsisr rX lwz rX, magic_page->dsisr
mtmsr rX std rX, magic_page->msr
mtsprg 0, rX std rX, magic_page->sprg0
mtsprg 1, rX std rX, magic_page->sprg1
mtsprg 2, rX std rX, magic_page->sprg2
mtsprg 3, rX std rX, magic_page->sprg3
mtsrr0 rX std rX, magic_page->srr0
mtsrr1 rX std rX, magic_page->srr1
mtdar rX std rX, magic_page->dar
mtdsisr rX stw rX, magic_page->dsisr
tlbsync nop
mtmsrd rX, 0 b <special mtmsr section>
mtmsr rX b <special mtmsr section>
mtmsrd rX, 1 b <special mtmsrd section>
[Book3S only]
mtsrin rX, rY b <special mtsrin section>
[BookE only]
wrteei [0|1] b <special wrteei section>
Some instructions require more logic to determine what's going on than a load
or store instruction can deliver. To enable patching of those, we keep some
RAM around where we can live translate instructions to. What happens is the
following:
1) copy emulation code to memory
2) patch that code to fit the emulated instruction
3) patch that code to return to the original pc + 4
4) patch the original instruction to branch to the new code
That way we can inject an arbitrary amount of code as replacement for a single
instruction. This allows us to check for pending interrupts when setting EE=1
for example.
Documentation/kvm/timekeeping.txt 0 → 100644
View file @ 1765a1fe
This diff is collapsed.
arch/ia64/kvm/lapic.h
View file @ 1765a1fe
......@@ -25,5 +25,6 @@ int kvm_apic_match_dest(struct kvm_vcpu *vcpu, struct kvm_lapic *source,
int kvm_apic_compare_prio(struct kvm_vcpu *vcpu1, struct kvm_vcpu *vcpu2);
int kvm_apic_set_irq(struct kvm_vcpu *vcpu, struct kvm_lapic_irq *irq);
#define kvm_apic_present(x) (true)
#define kvm_lapic_enabled(x) (true)
#endif
arch/powerpc/include/asm/kvm.h
View file @ 1765a1fe
......@@ -86,5 +86,6 @@ struct kvm_guest_debug_arch {
#define KVM_INTERRUPT_SET -1U
#define KVM_INTERRUPT_UNSET -2U
#define KVM_INTERRUPT_SET_LEVEL -3U
#endif /* __LINUX_KVM_POWERPC_H */
arch/powerpc/include/asm/kvm_asm.h
View file @ 1765a1fe
......@@ -58,6 +58,7 @@
#define BOOK3S_INTERRUPT_INST_STORAGE 0x400
#define BOOK3S_INTERRUPT_INST_SEGMENT 0x480
#define BOOK3S_INTERRUPT_EXTERNAL 0x500
#define BOOK3S_INTERRUPT_EXTERNAL_LEVEL 0x501
#define BOOK3S_INTERRUPT_ALIGNMENT 0x600
#define BOOK3S_INTERRUPT_PROGRAM 0x700
#define BOOK3S_INTERRUPT_FP_UNAVAIL 0x800
......@@ -84,7 +85,8 @@
#define BOOK3S_IRQPRIO_EXTERNAL 13
#define BOOK3S_IRQPRIO_DECREMENTER 14
#define BOOK3S_IRQPRIO_PERFORMANCE_MONITOR 15
#define BOOK3S_IRQPRIO_MAX 16
#define BOOK3S_IRQPRIO_EXTERNAL_LEVEL 16
#define BOOK3S_IRQPRIO_MAX 17
#define BOOK3S_HFLAG_DCBZ32 0x1
#define BOOK3S_HFLAG_SLB 0x2
......
arch/powerpc/include/asm/kvm_book3s.h
View file @ 1765a1fe
......@@ -38,15 +38,6 @@ struct kvmppc_slb {
bool class : 1;
};
struct kvmppc_sr {
u32 raw;
u32 vsid;
bool Ks : 1;
bool Kp : 1;
bool nx : 1;
bool valid : 1;
};
struct kvmppc_bat {
u64 raw;
u32 bepi;
......@@ -69,6 +60,13 @@ struct kvmppc_sid_map {
#define SID_MAP_NUM (1 << SID_MAP_BITS)
#define SID_MAP_MASK (SID_MAP_NUM - 1)
#ifdef CONFIG_PPC_BOOK3S_64
#define SID_CONTEXTS 1
#else
#define SID_CONTEXTS 128
#define VSID_POOL_SIZE (SID_CONTEXTS * 16)
#endif
struct kvmppc_vcpu_book3s {
struct kvm_vcpu vcpu;
struct kvmppc_book3s_shadow_vcpu *shadow_vcpu;
......@@ -79,20 +77,22 @@ struct kvmppc_vcpu_book3s {
u64 vsid;
} slb_shadow[64];
u8 slb_shadow_max;
struct kvmppc_sr sr[16];
struct kvmppc_bat ibat[8];
struct kvmppc_bat dbat[8];
u64 hid[6];
u64 gqr[8];
int slb_nr;
u32 dsisr;
u64 sdr1;
u64 hior;
u64 msr_mask;
u64 vsid_first;
u64 vsid_next;
#ifdef CONFIG_PPC_BOOK3S_32
u32 vsid_pool[VSID_POOL_SIZE];
#else
u64 vsid_first;
u64 vsid_max;
int context_id;
#endif
int context_id[SID_CONTEXTS];
ulong prog_flags; /* flags to inject when giving a 700 trap */
};
......@@ -131,9 +131,10 @@ extern void kvmppc_set_bat(struct kvm_vcpu *vcpu, struct kvmppc_bat *bat,
bool upper, u32 val);
extern void kvmppc_giveup_ext(struct kvm_vcpu *vcpu, ulong msr);
extern int kvmppc_emulate_paired_single(struct kvm_run *run, struct kvm_vcpu *vcpu);
extern pfn_t kvmppc_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn);
extern u32 kvmppc_trampoline_lowmem;
extern u32 kvmppc_trampoline_enter;
extern ulong kvmppc_trampoline_lowmem;
extern ulong kvmppc_trampoline_enter;
extern void kvmppc_rmcall(ulong srr0, ulong srr1);
extern void kvmppc_load_up_fpu(void);
extern void kvmppc_load_up_altivec(void);
......
arch/powerpc/include/asm/kvm_host.h
View file @ 1765a1fe
......@@ -25,6 +25,7 @@
#include <linux/interrupt.h>
#include <linux/types.h>
#include <linux/kvm_types.h>
#include <linux/kvm_para.h>
#include <asm/kvm_asm.h>
#define KVM_MAX_VCPUS 1
......@@ -41,12 +42,17 @@
#define HPTEG_CACHE_NUM (1 << 15)
#define HPTEG_HASH_BITS_PTE 13
#define HPTEG_HASH_BITS_PTE_LONG 12
#define HPTEG_HASH_BITS_VPTE 13
#define HPTEG_HASH_BITS_VPTE_LONG 5
#define HPTEG_HASH_NUM_PTE (1 << HPTEG_HASH_BITS_PTE)
#define HPTEG_HASH_NUM_PTE_LONG (1 << HPTEG_HASH_BITS_PTE_LONG)
#define HPTEG_HASH_NUM_VPTE (1 << HPTEG_HASH_BITS_VPTE)
#define HPTEG_HASH_NUM_VPTE_LONG (1 << HPTEG_HASH_BITS_VPTE_LONG)
/* Physical Address Mask - allowed range of real mode RAM access */
#define KVM_PAM 0x0fffffffffffffffULL
struct kvm;
struct kvm_run;
struct kvm_vcpu;
......@@ -159,8 +165,10 @@ struct kvmppc_mmu {
struct hpte_cache {
struct hlist_node list_pte;
struct hlist_node list_pte_long;
struct hlist_node list_vpte;
struct hlist_node list_vpte_long;
struct rcu_head rcu_head;
u64 host_va;
u64 pfn;
ulong slot;
......@@ -210,28 +218,20 @@ struct kvm_vcpu_arch {
u32 cr;
#endif
ulong msr;
#ifdef CONFIG_PPC_BOOK3S
ulong shadow_msr;
ulong hflags;
ulong guest_owned_ext;
#endif
u32 mmucr;
ulong sprg0;
ulong sprg1;
ulong sprg2;
ulong sprg3;
ulong sprg4;
ulong sprg5;
ulong sprg6;
ulong sprg7;
ulong srr0;
ulong srr1;
ulong csrr0;
ulong csrr1;
ulong dsrr0;
ulong dsrr1;
ulong dear;
ulong esr;
u32 dec;
u32 decar;
......@@ -290,12 +290,17 @@ struct kvm_vcpu_arch {
struct tasklet_struct tasklet;
u64 dec_jiffies;
unsigned long pending_exceptions;
struct kvm_vcpu_arch_shared *shared;
unsigned long magic_page_pa; /* phys addr to map the magic page to */
unsigned long magic_page_ea; /* effect. addr to map the magic page to */
#ifdef CONFIG_PPC_BOOK3S
struct hlist_head hpte_hash_pte[HPTEG_HASH_NUM_PTE];
struct hlist_head hpte_hash_pte_long[HPTEG_HASH_NUM_PTE_LONG];
struct hlist_head hpte_hash_vpte[HPTEG_HASH_NUM_VPTE];
struct hlist_head hpte_hash_vpte_long[HPTEG_HASH_NUM_VPTE_LONG];
int hpte_cache_count;
spinlock_t mmu_lock;
#endif
};
......
arch/powerpc/include/asm/kvm_para.h
View file @ 1765a1fe
......@@ -20,16 +20,153 @@
#ifndef __POWERPC_KVM_PARA_H__
#define __POWERPC_KVM_PARA_H__
#include <linux/types.h>
struct kvm_vcpu_arch_shared {
__u64 scratch1;
__u64 scratch2;
__u64 scratch3;
__u64 critical; /* Guest may not get interrupts if == r1 */
__u64 sprg0;
__u64 sprg1;
__u64 sprg2;
__u64 sprg3;
__u64 srr0;
__u64 srr1;
__u64 dar;
__u64 msr;
__u32 dsisr;
__u32 int_pending; /* Tells the guest if we have an interrupt */
__u32 sr[16];
};
#define KVM_SC_MAGIC_R0 0x4b564d21 /* "KVM!" */
#define HC_VENDOR_KVM (42 << 16)
#define HC_EV_SUCCESS 0
#define HC_EV_UNIMPLEMENTED 12
#define KVM_FEATURE_MAGIC_PAGE 1
#define KVM_MAGIC_FEAT_SR (1 << 0)
#ifdef __KERNEL__
#ifdef CONFIG_KVM_GUEST
#include <linux/of.h>
static inline int kvm_para_available(void)
{
struct device_node *hyper_node;
hyper_node = of_find_node_by_path("/hypervisor");
if (!hyper_node)
return 0;
if (!of_device_is_compatible(hyper_node, "linux,kvm"))
return 0;
return 1;
}
extern unsigned long kvm_hypercall(unsigned long *in,
unsigned long *out,
unsigned long nr);
#else
static inline int kvm_para_available(void)
{
return 0;
}
static unsigned long kvm_hypercall(unsigned long *in,
unsigned long *out,
unsigned long nr)
{
return HC_EV_UNIMPLEMENTED;
}
#endif
static inline long kvm_hypercall0_1(unsigned int nr, unsigned long *r2)
{
unsigned long in[8];
unsigned long out[8];
unsigned long r;
r = kvm_hypercall(in, out, nr | HC_VENDOR_KVM);
*r2 = out[0];
return r;
}
static inline long kvm_hypercall0(unsigned int nr)
{
unsigned long in[8];
unsigned long out[8];
return kvm_hypercall(in, out, nr | HC_VENDOR_KVM);
}
static inline long kvm_hypercall1(unsigned int nr, unsigned long p1)
{
unsigned long in[8];
unsigned long out[8];
in[0] = p1;
return kvm_hypercall(in, out, nr | HC_VENDOR_KVM);
}
static inline long kvm_hypercall2(unsigned int nr, unsigned long p1,
unsigned long p2)
{
unsigned long in[8];
unsigned long out[8];
in[0] = p1;
in[1] = p2;
return kvm_hypercall(in, out, nr | HC_VENDOR_KVM);
}
static inline long kvm_hypercall3(unsigned int nr, unsigned long p1,
unsigned long p2, unsigned long p3)
{
unsigned long in[8];
unsigned long out[8];
in[0] = p1;
in[1] = p2;
in[2] = p3;
return kvm_hypercall(in, out, nr | HC_VENDOR_KVM);
}
static inline long kvm_hypercall4(unsigned int nr, unsigned long p1,
unsigned long p2, unsigned long p3,
unsigned long p4)
{
unsigned long in[8];
unsigned long out[8];
in[0] = p1;
in[1] = p2;
in[2] = p3;
in[3] = p4;
return kvm_hypercall(in, out, nr | HC_VENDOR_KVM);
}
static inline unsigned int kvm_arch_para_features(void)
{
unsigned long r;
if (!kvm_para_available())
return 0;
if(kvm_hypercall0_1(KVM_HC_FEATURES, &r))
return 0;
return r;
}
#endif /* __KERNEL__ */
......
arch/powerpc/include/asm/kvm_ppc.h
View file @ 1765a1fe
......@@ -107,6 +107,7 @@ extern int kvmppc_booke_init(void);
extern void kvmppc_booke_exit(void);
extern void kvmppc_core_destroy_mmu(struct kvm_vcpu *vcpu);
extern int kvmppc_kvm_pv(struct kvm_vcpu *vcpu);
/*
* Cuts out inst bits with ordering according to spec.
......
arch/powerpc/kernel/Makefile
View file @ 1765a1fe
......@@ -129,6 +129,8 @@ ifneq ($(CONFIG_XMON)$(CONFIG_KEXEC),)
obj-y += ppc_save_regs.o
endif
obj-$(CONFIG_KVM_GUEST) += kvm.o kvm_emul.o
# Disable GCOV in odd or sensitive code
GCOV_PROFILE_prom_init.o := n
GCOV_PROFILE_ftrace.o := n
......
arch/powerpc/kernel/asm-offsets.c
View file @ 1765a1fe
......@@ -48,11 +48,11 @@
#ifdef CONFIG_PPC_ISERIES
#include <asm/iseries/alpaca.h>
#endif
#ifdef CONFIG_KVM
#if defined(CONFIG_KVM) || defined(CONFIG_KVM_GUEST)
#include <linux/kvm_host.h>
#ifndef CONFIG_BOOKE
#include <asm/kvm_book3s.h>
#endif
#if defined(CONFIG_KVM) && defined(CONFIG_PPC_BOOK3S)
#include <asm/kvm_book3s.h>
#endif
#ifdef CONFIG_PPC32
......@@ -396,12 +396,13 @@ int main(void)
DEFINE(VCPU_HOST_STACK, offsetof(struct kvm_vcpu, arch.host_stack));
DEFINE(VCPU_HOST_PID, offsetof(struct kvm_vcpu, arch.host_pid));
DEFINE(VCPU_GPRS, offsetof(struct kvm_vcpu, arch.gpr));
DEFINE(VCPU_MSR, offsetof(struct kvm_vcpu, arch.msr));
DEFINE(VCPU_SPRG4, offsetof(struct kvm_vcpu, arch.sprg4));
DEFINE(VCPU_SPRG5, offsetof(struct kvm_vcpu, arch.sprg5));
DEFINE(VCPU_SPRG6, offsetof(struct kvm_vcpu, arch.sprg6));
DEFINE(VCPU_SPRG7, offsetof(struct kvm_vcpu, arch.sprg7));
DEFINE(VCPU_SHADOW_PID, offsetof(struct kvm_vcpu, arch.shadow_pid));
DEFINE(VCPU_SHARED, offsetof(struct kvm_vcpu, arch.shared));
DEFINE(VCPU_SHARED_MSR, offsetof(struct kvm_vcpu_arch_shared, msr));
/* book3s */
#ifdef CONFIG_PPC_BOOK3S
......@@ -466,6 +467,22 @@ int main(void)
DEFINE(VCPU_FAULT_ESR, offsetof(struct kvm_vcpu, arch.fault_esr));
#endif /* CONFIG_PPC_BOOK3S */
#endif
#ifdef CONFIG_KVM_GUEST
DEFINE(KVM_MAGIC_SCRATCH1, offsetof(struct kvm_vcpu_arch_shared,
scratch1));
DEFINE(KVM_MAGIC_SCRATCH2, offsetof(struct kvm_vcpu_arch_shared,
scratch2));
DEFINE(KVM_MAGIC_SCRATCH3, offsetof(struct kvm_vcpu_arch_shared,
scratch3));
DEFINE(KVM_MAGIC_INT, offsetof(struct kvm_vcpu_arch_shared,
int_pending));
DEFINE(KVM_MAGIC_MSR, offsetof(struct kvm_vcpu_arch_shared, msr));
DEFINE(KVM_MAGIC_CRITICAL, offsetof(struct kvm_vcpu_arch_shared,
critical));
DEFINE(KVM_MAGIC_SR, offsetof(struct kvm_vcpu_arch_shared, sr));
#endif
#ifdef CONFIG_44x
DEFINE(PGD_T_LOG2, PGD_T_LOG2);
DEFINE(PTE_T_LOG2, PTE_T_LOG2);
......
arch/powerpc/kernel/exceptions-64s.S
View file @ 1765a1fe
......@@ -299,6 +299,12 @@ slb_miss_user_pseries:
b . /* prevent spec. execution */
#endif /* __DISABLED__ */
/* KVM's trampoline code needs to be close to the interrupt handlers */
#ifdef CONFIG_KVM_BOOK3S_64_HANDLER
#include "../kvm/book3s_rmhandlers.S"
#endif
.align 7
.globl __end_interrupts
__end_interrupts:
......
arch/powerpc/kernel/head_64.S
View file @ 1765a1fe
......@@ -166,12 +166,6 @@ exception_marker:
#include "exceptions-64s.S"
#endif
/* KVM trampoline code needs to be close to the interrupt handlers */
#ifdef CONFIG_KVM_BOOK3S_64_HANDLER
#include "../kvm/book3s_rmhandlers.S"
#endif
_GLOBAL(generic_secondary_thread_init)
mr r24,r3
......
arch/powerpc/kernel/kvm.c 0 → 100644
View file @ 1765a1fe
This diff is collapsed.
arch/powerpc/kernel/kvm_emul.S 0 → 100644
View file @ 1765a1fe
/*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License, version 2, as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* Copyright SUSE Linux Products GmbH 2010
*
* Authors: Alexander Graf <agraf@suse.de>
*/
#include <asm/ppc_asm.h>
#include <asm/kvm_asm.h>
#include <asm/reg.h>
#include <asm/page.h>
#include <asm/asm-offsets.h>
/* Hypercall entry point. Will be patched with device tree instructions. */
.global kvm_hypercall_start
kvm_hypercall_start:
li r3, -1
nop
nop
nop
blr
#define KVM_MAGIC_PAGE (-4096)
#ifdef CONFIG_64BIT
#define LL64(reg, offs, reg2) ld reg, (offs)(reg2)
#define STL64(reg, offs, reg2) std reg, (offs)(reg2)
#else
#define LL64(reg, offs, reg2) lwz reg, (offs + 4)(reg2)
#define STL64(reg, offs, reg2) stw reg, (offs + 4)(reg2)
#endif
#define SCRATCH_SAVE \
/* Enable critical section. We are critical if \
shared->critical == r1 */ \
STL64(r1, KVM_MAGIC_PAGE + KVM_MAGIC_CRITICAL, 0); \
\
/* Save state */ \
PPC_STL r31, (KVM_MAGIC_PAGE + KVM_MAGIC_SCRATCH1)(0); \
PPC_STL r30, (KVM_MAGIC_PAGE + KVM_MAGIC_SCRATCH2)(0); \
mfcr r31; \
stw r31, (KVM_MAGIC_PAGE + KVM_MAGIC_SCRATCH3)(0);
#define SCRATCH_RESTORE \
/* Restore state */ \
PPC_LL r31, (KVM_MAGIC_PAGE + KVM_MAGIC_SCRATCH1)(0); \
lwz r30, (KVM_MAGIC_PAGE + KVM_MAGIC_SCRATCH3)(0); \
mtcr r30; \
PPC_LL r30, (KVM_MAGIC_PAGE + KVM_MAGIC_SCRATCH2)(0); \
\
/* Disable critical section. We are critical if \
shared->critical == r1 and r2 is always != r1 */ \
STL64(r2, KVM_MAGIC_PAGE + KVM_MAGIC_CRITICAL, 0);
.global kvm_emulate_mtmsrd
kvm_emulate_mtmsrd:
SCRATCH_SAVE
/* Put MSR & ~(MSR_EE|MSR_RI) in r31 */
LL64(r31, KVM_MAGIC_PAGE + KVM_MAGIC_MSR, 0)
lis r30, (~(MSR_EE | MSR_RI))@h
ori r30, r30, (~(MSR_EE | MSR_RI))@l
and r31, r31, r30
/* OR the register's (MSR_EE|MSR_RI) on MSR */
kvm_emulate_mtmsrd_reg:
ori r30, r0, 0
andi. r30, r30, (MSR_EE|MSR_RI)
or r31, r31, r30
/* Put MSR back into magic page */
STL64(r31, KVM_MAGIC_PAGE + KVM_MAGIC_MSR, 0)
/* Check if we have to fetch an interrupt */
lwz r31, (KVM_MAGIC_PAGE + KVM_MAGIC_INT)(0)
cmpwi r31, 0
beq+ no_check
/* Check if we may trigger an interrupt */
andi. r30, r30, MSR_EE
beq no_check
SCRATCH_RESTORE
/* Nag hypervisor */
kvm_emulate_mtmsrd_orig_ins:
tlbsync
b kvm_emulate_mtmsrd_branch
no_check:
SCRATCH_RESTORE
/* Go back to caller */
kvm_emulate_mtmsrd_branch:
b .
kvm_emulate_mtmsrd_end:
.global kvm_emulate_mtmsrd_branch_offs
kvm_emulate_mtmsrd_branch_offs:
.long (kvm_emulate_mtmsrd_branch - kvm_emulate_mtmsrd) / 4
.global kvm_emulate_mtmsrd_reg_offs
kvm_emulate_mtmsrd_reg_offs:
.long (kvm_emulate_mtmsrd_reg - kvm_emulate_mtmsrd) / 4
.global kvm_emulate_mtmsrd_orig_ins_offs
kvm_emulate_mtmsrd_orig_ins_offs:
.long (kvm_emulate_mtmsrd_orig_ins - kvm_emulate_mtmsrd) / 4
.global kvm_emulate_mtmsrd_len
kvm_emulate_mtmsrd_len:
.long (kvm_emulate_mtmsrd_end - kvm_emulate_mtmsrd) / 4
#define MSR_SAFE_BITS (MSR_EE | MSR_CE | MSR_ME | MSR_RI)
#define MSR_CRITICAL_BITS ~MSR_SAFE_BITS
.global kvm_emulate_mtmsr
kvm_emulate_mtmsr:
SCRATCH_SAVE
/* Fetch old MSR in r31 */
LL64(r31, KVM_MAGIC_PAGE + KVM_MAGIC_MSR, 0)
/* Find the changed bits between old and new MSR */
kvm_emulate_mtmsr_reg1:
ori r30, r0, 0
xor r31, r30, r31
/* Check if we need to really do mtmsr */
LOAD_REG_IMMEDIATE(r30, MSR_CRITICAL_BITS)
and. r31, r31, r30
/* No critical bits changed? Maybe we can stay in the guest. */
beq maybe_stay_in_guest
do_mtmsr:
SCRATCH_RESTORE
/* Just fire off the mtmsr if it's critical */
kvm_emulate_mtmsr_orig_ins:
mtmsr r0
b kvm_emulate_mtmsr_branch
maybe_stay_in_guest:
/* Get the target register in r30 */
kvm_emulate_mtmsr_reg2:
ori r30, r0, 0
/* Check if we have to fetch an interrupt */
lwz r31, (KVM_MAGIC_PAGE + KVM_MAGIC_INT)(0)
cmpwi r31, 0
beq+ no_mtmsr
/* Check if we may trigger an interrupt */
andi. r31, r30, MSR_EE
beq no_mtmsr
b do_mtmsr
no_mtmsr:
/* Put MSR into magic page because we don't call mtmsr */
STL64(r30, KVM_MAGIC_PAGE + KVM_MAGIC_MSR, 0)
SCRATCH_RESTORE
/* Go back to caller */
kvm_emulate_mtmsr_branch:
b .
kvm_emulate_mtmsr_end:
.global kvm_emulate_mtmsr_branch_offs
kvm_emulate_mtmsr_branch_offs:
.long (kvm_emulate_mtmsr_branch - kvm_emulate_mtmsr) / 4
.global kvm_emulate_mtmsr_reg1_offs
kvm_emulate_mtmsr_reg1_offs:
.long (kvm_emulate_mtmsr_reg1 - kvm_emulate_mtmsr) / 4
.global kvm_emulate_mtmsr_reg2_offs
kvm_emulate_mtmsr_reg2_offs:
.long (kvm_emulate_mtmsr_reg2 - kvm_emulate_mtmsr) / 4
.global kvm_emulate_mtmsr_orig_ins_offs
kvm_emulate_mtmsr_orig_ins_offs:
.long (kvm_emulate_mtmsr_orig_ins - kvm_emulate_mtmsr) / 4
.global kvm_emulate_mtmsr_len
kvm_emulate_mtmsr_len:
.long (kvm_emulate_mtmsr_end - kvm_emulate_mtmsr) / 4
.global kvm_emulate_wrteei
kvm_emulate_wrteei:
SCRATCH_SAVE
/* Fetch old MSR in r31 */
LL64(r31, KVM_MAGIC_PAGE + KVM_MAGIC_MSR, 0)
/* Remove MSR_EE from old MSR */
li r30, 0
ori r30, r30, MSR_EE
andc r31, r31, r30
/* OR new MSR_EE onto the old MSR */
kvm_emulate_wrteei_ee:
ori r31, r31, 0
/* Write new MSR value back */
STL64(r31, KVM_MAGIC_PAGE + KVM_MAGIC_MSR, 0)
SCRATCH_RESTORE
/* Go back to caller */
kvm_emulate_wrteei_branch:
b .
kvm_emulate_wrteei_end:
.global kvm_emulate_wrteei_branch_offs
kvm_emulate_wrteei_branch_offs:
.long (kvm_emulate_wrteei_branch - kvm_emulate_wrteei) / 4
.global kvm_emulate_wrteei_ee_offs
kvm_emulate_wrteei_ee_offs:
.long (kvm_emulate_wrteei_ee - kvm_emulate_wrteei) / 4
.global kvm_emulate_wrteei_len
kvm_emulate_wrteei_len:
.long (kvm_emulate_wrteei_end - kvm_emulate_wrteei) / 4
.global kvm_emulate_mtsrin
kvm_emulate_mtsrin:
SCRATCH_SAVE
LL64(r31, KVM_MAGIC_PAGE + KVM_MAGIC_MSR, 0)
andi. r31, r31, MSR_DR | MSR_IR
beq kvm_emulate_mtsrin_reg1
SCRATCH_RESTORE
kvm_emulate_mtsrin_orig_ins:
nop
b kvm_emulate_mtsrin_branch
kvm_emulate_mtsrin_reg1:
/* rX >> 26 */
rlwinm r30,r0,6,26,29
kvm_emulate_mtsrin_reg2:
stw r0, (KVM_MAGIC_PAGE + KVM_MAGIC_SR)(r30)
SCRATCH_RESTORE
/* Go back to caller */
kvm_emulate_mtsrin_branch:
b .
kvm_emulate_mtsrin_end:
.global kvm_emulate_mtsrin_branch_offs
kvm_emulate_mtsrin_branch_offs:
.long (kvm_emulate_mtsrin_branch - kvm_emulate_mtsrin) / 4
.global kvm_emulate_mtsrin_reg1_offs
kvm_emulate_mtsrin_reg1_offs:
.long (kvm_emulate_mtsrin_reg1 - kvm_emulate_mtsrin) / 4
.global kvm_emulate_mtsrin_reg2_offs
kvm_emulate_mtsrin_reg2_offs:
.long (kvm_emulate_mtsrin_reg2 - kvm_emulate_mtsrin) / 4
.global kvm_emulate_mtsrin_orig_ins_offs
kvm_emulate_mtsrin_orig_ins_offs:
.long (kvm_emulate_mtsrin_orig_ins - kvm_emulate_mtsrin) / 4
.global kvm_emulate_mtsrin_len
kvm_emulate_mtsrin_len:
.long (kvm_emulate_mtsrin_end - kvm_emulate_mtsrin) / 4
arch/powerpc/kvm/44x.c
View file @ 1765a1fe
......@@ -43,7 +43,7 @@ int kvmppc_core_check_processor_compat(void)
{
int r;
if (strcmp(cur_cpu_spec->platform, "ppc440") == 0)
if (strncmp(cur_cpu_spec->platform, "ppc440", 6) == 0)
r = 0;
else
r = -ENOTSUPP;
......@@ -72,6 +72,7 @@ int kvmppc_core_vcpu_setup(struct kvm_vcpu *vcpu)
/* Since the guest can directly access the timebase, it must know the
* real timebase frequency. Accordingly, it must see the state of
* CCR1[TCS]. */
/* XXX CCR1 doesn't exist on all 440 SoCs. */
vcpu->arch.ccr1 = mfspr(SPRN_CCR1);
for (i = 0; i < ARRAY_SIZE(vcpu_44x->shadow_refs); i++)
......@@ -123,8 +124,14 @@ struct kvm_vcpu *kvmppc_core_vcpu_create(struct kvm *kvm, unsigned int id)
if (err)
goto free_vcpu;
vcpu->arch.shared = (void*)__get_free_page(GFP_KERNEL|__GFP_ZERO);
if (!vcpu->arch.shared)
goto uninit_vcpu;
return vcpu;
uninit_vcpu:
kvm_vcpu_uninit(vcpu);
free_vcpu:
kmem_cache_free(kvm_vcpu_cache, vcpu_44x);
out:
......@@ -135,6 +142,7 @@ void kvmppc_core_vcpu_free(struct kvm_vcpu *vcpu)
{
struct kvmppc_vcpu_44x *vcpu_44x = to_44x(vcpu);
free_page((unsigned long)vcpu->arch.shared);
kvm_vcpu_uninit(vcpu);
kmem_cache_free(kvm_vcpu_cache, vcpu_44x);
}
......
arch/powerpc/kvm/44x_tlb.c
View file @ 1765a1fe
......@@ -47,6 +47,7 @@
#ifdef DEBUG
void kvmppc_dump_tlbs(struct kvm_vcpu *vcpu)
{
struct kvmppc_vcpu_44x *vcpu_44x = to_44x(vcpu);
struct kvmppc_44x_tlbe *tlbe;
int i;
......@@ -221,14 +222,14 @@ gpa_t kvmppc_mmu_xlate(struct kvm_vcpu *vcpu, unsigned int gtlb_index,
int kvmppc_mmu_itlb_index(struct kvm_vcpu *vcpu, gva_t eaddr)
{
unsigned int as = !!(vcpu->arch.msr & MSR_IS);
unsigned int as = !!(vcpu->arch.shared->msr & MSR_IS);
return kvmppc_44x_tlb_index(vcpu, eaddr, vcpu->arch.pid, as);
}
int kvmppc_mmu_dtlb_index(struct kvm_vcpu *vcpu, gva_t eaddr)
{
unsigned int as = !!(vcpu->arch.msr & MSR_DS);
unsigned int as = !!(vcpu->arch.shared->msr & MSR_DS);
return kvmppc_44x_tlb_index(vcpu, eaddr, vcpu->arch.pid, as);
}
......@@ -354,7 +355,7 @@ void kvmppc_mmu_map(struct kvm_vcpu *vcpu, u64 gvaddr, gpa_t gpaddr,
stlbe.word1 = (hpaddr & 0xfffffc00) | ((hpaddr >> 32) & 0xf);
stlbe.word2 = kvmppc_44x_tlb_shadow_attrib(flags,
vcpu->arch.msr & MSR_PR);
vcpu->arch.shared->msr & MSR_PR);
stlbe.tid = !(asid & 0xff);
/* Keep track of the reference so we can properly release it later. */
......@@ -423,7 +424,7 @@ static int tlbe_is_host_safe(const struct kvm_vcpu *vcpu,
/* Does it match current guest AS? */
/* XXX what about IS != DS? */
if (get_tlb_ts(tlbe) != !!(vcpu->arch.msr & MSR_IS))
if (get_tlb_ts(tlbe) != !!(vcpu->arch.shared->msr & MSR_IS))
return 0;
gpa = get_tlb_raddr(tlbe);
......
arch/powerpc/kvm/book3s.c
View file @ 1765a1fe
This diff is collapsed.
arch/powerpc/kvm/book3s_32_mmu.c
View file @ 1765a1fe
......@@ -58,14 +58,39 @@ static inline bool check_debug_ip(struct kvm_vcpu *vcpu)
#endif
}
static inline u32 sr_vsid(u32 sr_raw)
{
return sr_raw & 0x0fffffff;
}
static inline bool sr_valid(u32 sr_raw)
{
return (sr_raw & 0x80000000) ? false : true;
}
static inline bool sr_ks(u32 sr_raw)
{
return (sr_raw & 0x40000000) ? true: false;
}
static inline bool sr_kp(u32 sr_raw)
{
return (sr_raw & 0x20000000) ? true: false;
}
static inline bool sr_nx(u32 sr_raw)
{
return (sr_raw & 0x10000000) ? true: false;
}
static int kvmppc_mmu_book3s_32_xlate_bat(struct kvm_vcpu *vcpu, gva_t eaddr,
struct kvmppc_pte *pte, bool data);
static int kvmppc_mmu_book3s_32_esid_to_vsid(struct kvm_vcpu *vcpu, ulong esid,
u64 *vsid);
static struct kvmppc_sr *find_sr(struct kvmppc_vcpu_book3s *vcpu_book3s, gva_t eaddr)
static u32 find_sr(struct kvm_vcpu *vcpu, gva_t eaddr)
{
return &vcpu_book3s->sr[(eaddr >> 28) & 0xf];
return vcpu->arch.shared->sr[(eaddr >> 28) & 0xf];
}
static u64 kvmppc_mmu_book3s_32_ea_to_vp(struct kvm_vcpu *vcpu, gva_t eaddr,
......@@ -87,7 +112,7 @@ static void kvmppc_mmu_book3s_32_reset_msr(struct kvm_vcpu *vcpu)
}
static hva_t kvmppc_mmu_book3s_32_get_pteg(struct kvmppc_vcpu_book3s *vcpu_book3s,
struct kvmppc_sr *sre, gva_t eaddr,
u32 sre, gva_t eaddr,
bool primary)
{
u32 page, hash, pteg, htabmask;
......@@ -96,7 +121,7 @@ static hva_t kvmppc_mmu_book3s_32_get_pteg(struct kvmppc_vcpu_book3s *vcpu_book3
page = (eaddr & 0x0FFFFFFF) >> 12;
htabmask = ((vcpu_book3s->sdr1 & 0x1FF) << 16) | 0xFFC0;
hash = ((sre->vsid ^ page) << 6);
hash = ((sr_vsid(sre) ^ page) << 6);
if (!primary)
hash = ~hash;
hash &= htabmask;
......@@ -104,8 +129,8 @@ static hva_t kvmppc_mmu_book3s_32_get_pteg(struct kvmppc_vcpu_book3s *vcpu_book3
pteg = (vcpu_book3s->sdr1 & 0xffff0000) | hash;
dprintk("MMU: pc=0x%lx eaddr=0x%lx sdr1=0x%llx pteg=0x%x vsid=0x%x\n",
vcpu_book3s->vcpu.arch.pc, eaddr, vcpu_book3s->sdr1, pteg,
sre->vsid);
kvmppc_get_pc(&vcpu_book3s->vcpu), eaddr, vcpu_book3s->sdr1, pteg,
sr_vsid(sre));
r = gfn_to_hva(vcpu_book3s->vcpu.kvm, pteg >> PAGE_SHIFT);
if (kvm_is_error_hva(r))
......@@ -113,10 +138,9 @@ static hva_t kvmppc_mmu_book3s_32_get_pteg(struct kvmppc_vcpu_book3s *vcpu_book3
return r | (pteg & ~PAGE_MASK);
}
static u32 kvmppc_mmu_book3s_32_get_ptem(struct kvmppc_sr *sre, gva_t eaddr,
bool primary)
static u32 kvmppc_mmu_book3s_32_get_ptem(u32 sre, gva_t eaddr, bool primary)
{
return ((eaddr & 0x0fffffff) >> 22) | (sre->vsid << 7) |
return ((eaddr & 0x0fffffff) >> 22) | (sr_vsid(sre) << 7) |
(primary ? 0 : 0x40) | 0x80000000;
}
......@@ -133,7 +157,7 @@ static int kvmppc_mmu_book3s_32_xlate_bat(struct kvm_vcpu *vcpu, gva_t eaddr,
else
bat = &vcpu_book3s->ibat[i];
if (vcpu->arch.msr & MSR_PR) {
if (vcpu->arch.shared->msr & MSR_PR) {
if (!bat->vp)
continue;
} else {
......@@ -180,17 +204,17 @@ static int kvmppc_mmu_book3s_32_xlate_pte(struct kvm_vcpu *vcpu, gva_t eaddr,
bool primary)
{
struct kvmppc_vcpu_book3s *vcpu_book3s = to_book3s(vcpu);
struct kvmppc_sr *sre;
u32 sre;
hva_t ptegp;
u32 pteg[16];
u32 ptem = 0;
int i;
int found = 0;
sre = find_sr(vcpu_book3s, eaddr);
sre = find_sr(vcpu, eaddr);
dprintk_pte("SR 0x%lx: vsid=0x%x, raw=0x%x\n", eaddr >> 28,
sre->vsid, sre->raw);
sr_vsid(sre), sre);
pte->vpage = kvmppc_mmu_book3s_32_ea_to_vp(vcpu, eaddr, data);
......@@ -214,8 +238,8 @@ static int kvmppc_mmu_book3s_32_xlate_pte(struct kvm_vcpu *vcpu, gva_t eaddr,
pte->raddr = (pteg[i+1] & ~(0xFFFULL)) | (eaddr & 0xFFF);
pp = pteg[i+1] & 3;
if ((sre->Kp && (vcpu->arch.msr & MSR_PR)) ||
(sre->Ks && !(vcpu->arch.msr & MSR_PR)))
if ((sr_kp(sre) && (vcpu->arch.shared->msr & MSR_PR)) ||
(sr_ks(sre) && !(vcpu->arch.shared->msr & MSR_PR)))
pp |= 4;
pte->may_write = false;
......@@ -269,7 +293,7 @@ static int kvmppc_mmu_book3s_32_xlate_pte(struct kvm_vcpu *vcpu, gva_t eaddr,
dprintk_pte("KVM MMU: No PTE found (sdr1=0x%llx ptegp=0x%lx)\n",
to_book3s(vcpu)->sdr1, ptegp);
for (i=0; i<16; i+=2) {
dprintk_pte(" %02d: 0x%x - 0x%x (0x%llx)\n",
dprintk_pte(" %02d: 0x%x - 0x%x (0x%x)\n",
i, pteg[i], pteg[i+1], ptem);
}
}
......@@ -281,8 +305,24 @@ static int kvmppc_mmu_book3s_32_xlate(struct kvm_vcpu *vcpu, gva_t eaddr,
struct kvmppc_pte *pte, bool data)
{
int r;
ulong mp_ea = vcpu->arch.magic_page_ea;
pte->eaddr = eaddr;
/* Magic page override */
if (unlikely(mp_ea) &&
unlikely((eaddr & ~0xfffULL) == (mp_ea & ~0xfffULL)) &&
!(vcpu->arch.shared->msr & MSR_PR)) {
pte->vpage = kvmppc_mmu_book3s_32_ea_to_vp(vcpu, eaddr, data);
pte->raddr = vcpu->arch.magic_page_pa | (pte->raddr & 0xfff);
pte->raddr &= KVM_PAM;
pte->may_execute = true;
pte->may_read = true;
pte->may_write = true;
return 0;
}
r = kvmppc_mmu_book3s_32_xlate_bat(vcpu, eaddr, pte, data);
if (r < 0)
r = kvmppc_mmu_book3s_32_xlate_pte(vcpu, eaddr, pte, data, true);
......@@ -295,30 +335,13 @@ static int kvmppc_mmu_book3s_32_xlate(struct kvm_vcpu *vcpu, gva_t eaddr,
static u32 kvmppc_mmu_book3s_32_mfsrin(struct kvm_vcpu *vcpu, u32 srnum)
{
return to_book3s(vcpu)->sr[srnum].raw;
return vcpu->arch.shared->sr[srnum];
}
static void kvmppc_mmu_book3s_32_mtsrin(struct kvm_vcpu *vcpu, u32 srnum,
ulong value)
{
struct kvmppc_sr *sre;
sre = &to_book3s(vcpu)->sr[srnum];
/* Flush any left-over shadows from the previous SR */
/* XXX Not necessary? */
/* kvmppc_mmu_pte_flush(vcpu, ((u64)sre->vsid) << 28, 0xf0000000ULL); */
/* And then put in the new SR */
sre->raw = value;
sre->vsid = (value & 0x0fffffff);
sre->valid = (value & 0x80000000) ? false : true;
sre->Ks = (value & 0x40000000) ? true : false;
sre->Kp = (value & 0x20000000) ? true : false;
sre->nx = (value & 0x10000000) ? true : false;
/* Map the new segment */
vcpu->arch.shared->sr[srnum] = value;
kvmppc_mmu_map_segment(vcpu, srnum << SID_SHIFT);
}
......@@ -331,19 +354,19 @@ static int kvmppc_mmu_book3s_32_esid_to_vsid(struct kvm_vcpu *vcpu, ulong esid,
u64 *vsid)
{
ulong ea = esid << SID_SHIFT;
struct kvmppc_sr *sr;
u32 sr;
u64 gvsid = esid;
if (vcpu->arch.msr & (MSR_DR|MSR_IR)) {
sr = find_sr(to_book3s(vcpu), ea);
if (sr->valid)
gvsid = sr->vsid;
if (vcpu->arch.shared->msr & (MSR_DR|MSR_IR)) {
sr = find_sr(vcpu, ea);
if (sr_valid(sr))
gvsid = sr_vsid(sr);
}
/* In case we only have one of MSR_IR or MSR_DR set, let's put
that in the real-mode context (and hope RM doesn't access
high memory) */
switch (vcpu->arch.msr & (MSR_DR|MSR_IR)) {
switch (vcpu->arch.shared->msr & (MSR_DR|MSR_IR)) {
case 0:
*vsid = VSID_REAL | esid;
break;
......@@ -354,8 +377,8 @@ static int kvmppc_mmu_book3s_32_esid_to_vsid(struct kvm_vcpu *vcpu, ulong esid,
*vsid = VSID_REAL_DR | gvsid;
break;
case MSR_DR|MSR_IR:
if (sr->valid)
*vsid = sr->vsid;
if (sr_valid(sr))
*vsid = sr_vsid(sr);
else
*vsid = VSID_BAT | gvsid;
break;
......@@ -363,7 +386,7 @@ static int kvmppc_mmu_book3s_32_esid_to_vsid(struct kvm_vcpu *vcpu, ulong esid,
BUG();
}
if (vcpu->arch.msr & MSR_PR)
if (vcpu->arch.shared->msr & MSR_PR)
*vsid |= VSID_PR;
return 0;
......
arch/powerpc/kvm/book3s_32_mmu_host.c
View file @ 1765a1fe
......@@ -19,7 +19,6 @@
*/
#include <linux/kvm_host.h>
#include <linux/hash.h>
#include <asm/kvm_ppc.h>
#include <asm/kvm_book3s.h>
......@@ -77,7 +76,14 @@ void kvmppc_mmu_invalidate_pte(struct kvm_vcpu *vcpu, struct hpte_cache *pte)
* a hash, so we don't waste cycles on looping */
static u16 kvmppc_sid_hash(struct kvm_vcpu *vcpu, u64 gvsid)
{
return hash_64(gvsid, SID_MAP_BITS);
return (u16)(((gvsid >> (SID_MAP_BITS * 7)) & SID_MAP_MASK) ^
((gvsid >> (SID_MAP_BITS * 6)) & SID_MAP_MASK) ^
((gvsid >> (SID_MAP_BITS * 5)) & SID_MAP_MASK) ^
((gvsid >> (SID_MAP_BITS * 4)) & SID_MAP_MASK) ^
((gvsid >> (SID_MAP_BITS * 3)) & SID_MAP_MASK) ^
((gvsid >> (SID_MAP_BITS * 2)) & SID_MAP_MASK) ^
((gvsid >> (SID_MAP_BITS * 1)) & SID_MAP_MASK) ^
((gvsid >> (SID_MAP_BITS * 0)) & SID_MAP_MASK));
}
......@@ -86,7 +92,7 @@ static struct kvmppc_sid_map *find_sid_vsid(struct kvm_vcpu *vcpu, u64 gvsid)
struct kvmppc_sid_map *map;
u16 sid_map_mask;
if (vcpu->arch.msr & MSR_PR)
if (vcpu->arch.shared->msr & MSR_PR)
gvsid |= VSID_PR;
sid_map_mask = kvmppc_sid_hash(vcpu, gvsid);
......@@ -147,8 +153,8 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte)
struct hpte_cache *pte;
/* Get host physical address for gpa */
hpaddr = gfn_to_pfn(vcpu->kvm, orig_pte->raddr >> PAGE_SHIFT);
if (kvm_is_error_hva(hpaddr)) {
hpaddr = kvmppc_gfn_to_pfn(vcpu, orig_pte->raddr >> PAGE_SHIFT);
if (is_error_pfn(hpaddr)) {
printk(KERN_INFO "Couldn't get guest page for gfn %lx!\n",
orig_pte->eaddr);
return -EINVAL;
......@@ -253,7 +259,7 @@ static struct kvmppc_sid_map *create_sid_map(struct kvm_vcpu *vcpu, u64 gvsid)
u16 sid_map_mask;
static int backwards_map = 0;
if (vcpu->arch.msr & MSR_PR)
if (vcpu->arch.shared->msr & MSR_PR)
gvsid |= VSID_PR;
/* We might get collisions that trap in preceding order, so let's
......@@ -269,18 +275,15 @@ static struct kvmppc_sid_map *create_sid_map(struct kvm_vcpu *vcpu, u64 gvsid)
backwards_map = !backwards_map;
/* Uh-oh ... out of mappings. Let's flush! */
if (vcpu_book3s->vsid_next >= vcpu_book3s->vsid_max) {
vcpu_book3s->vsid_next = vcpu_book3s->vsid_first;
if (vcpu_book3s->vsid_next >= VSID_POOL_SIZE) {
vcpu_book3s->vsid_next = 0;
memset(vcpu_book3s->sid_map, 0,
sizeof(struct kvmppc_sid_map) * SID_MAP_NUM);
kvmppc_mmu_pte_flush(vcpu, 0, 0);
kvmppc_mmu_flush_segments(vcpu);
}
map->host_vsid = vcpu_book3s->vsid_next;
/* Would have to be 111 to be completely aligned with the rest of
Linux, but that is just way too little space! */
vcpu_book3s->vsid_next+=1;
map->host_vsid = vcpu_book3s->vsid_pool[vcpu_book3s->vsid_next];
vcpu_book3s->vsid_next++;
map->guest_vsid = gvsid;
map->valid = true;
......@@ -327,40 +330,38 @@ void kvmppc_mmu_flush_segments(struct kvm_vcpu *vcpu)
void kvmppc_mmu_destroy(struct kvm_vcpu *vcpu)
{
int i;
kvmppc_mmu_hpte_destroy(vcpu);
preempt_disable();
__destroy_context(to_book3s(vcpu)->context_id);
for (i = 0; i < SID_CONTEXTS; i++)
__destroy_context(to_book3s(vcpu)->context_id[i]);
preempt_enable();
}
/* From mm/mmu_context_hash32.c */
#define CTX_TO_VSID(ctx) (((ctx) * (897 * 16)) & 0xffffff)
#define CTX_TO_VSID(c, id) ((((c) * (897 * 16)) + (id * 0x111)) & 0xffffff)
int kvmppc_mmu_init(struct kvm_vcpu *vcpu)
{
struct kvmppc_vcpu_book3s *vcpu3s = to_book3s(vcpu);
int err;
ulong sdr1;
int i;
int j;
for (i = 0; i < SID_CONTEXTS; i++) {
err = __init_new_context();
if (err < 0)
return -1;
vcpu3s->context_id = err;
vcpu3s->vsid_max = CTX_TO_VSID(vcpu3s->context_id + 1) - 1;
vcpu3s->vsid_first = CTX_TO_VSID(vcpu3s->context_id);
#if 0 /* XXX still doesn't guarantee uniqueness */
/* We could collide with the Linux vsid space because the vsid
* wraps around at 24 bits. We're safe if we do our own space
* though, so let's always set the highest bit. */
goto init_fail;
vcpu3s->context_id[i] = err;
vcpu3s->vsid_max |= 0x00800000;
vcpu3s->vsid_first |= 0x00800000;
#endif
BUG_ON(vcpu3s->vsid_max < vcpu3s->vsid_first);
/* Remember context id for this combination */
for (j = 0; j < 16; j++)
vcpu3s->vsid_pool[(i * 16) + j] = CTX_TO_VSID(err, j);
}
vcpu3s->vsid_next = vcpu3s->vsid_first;
vcpu3s->vsid_next = 0;
/* Remember where the HTAB is */
asm ( "mfsdr1 %0" : "=r"(sdr1) );
......@@ -370,4 +371,14 @@ int kvmppc_mmu_init(struct kvm_vcpu *vcpu)
kvmppc_mmu_hpte_init(vcpu);
return 0;
init_fail:
for (j = 0; j < i; j++) {
if (!vcpu3s->context_id[j])
continue;
__destroy_context(to_book3s(vcpu)->context_id[j]);
}
return -1;
}
arch/powerpc/kvm/book3s_64_mmu.c
View file @ 1765a1fe
......@@ -163,6 +163,22 @@ static int kvmppc_mmu_book3s_64_xlate(struct kvm_vcpu *vcpu, gva_t eaddr,
bool found = false;
bool perm_err = false;
int second = 0;
ulong mp_ea = vcpu->arch.magic_page_ea;
/* Magic page override */
if (unlikely(mp_ea) &&
unlikely((eaddr & ~0xfffULL) == (mp_ea & ~0xfffULL)) &&
!(vcpu->arch.shared->msr & MSR_PR)) {
gpte->eaddr = eaddr;
gpte->vpage = kvmppc_mmu_book3s_64_ea_to_vp(vcpu, eaddr, data);
gpte->raddr = vcpu->arch.magic_page_pa | (gpte->raddr & 0xfff);
gpte->raddr &= KVM_PAM;
gpte->may_execute = true;
gpte->may_read = true;
gpte->may_write = true;
return 0;
}
slbe = kvmppc_mmu_book3s_64_find_slbe(vcpu_book3s, eaddr);
if (!slbe)
......@@ -180,9 +196,9 @@ static int kvmppc_mmu_book3s_64_xlate(struct kvm_vcpu *vcpu, gva_t eaddr,
goto no_page_found;
}
if ((vcpu->arch.msr & MSR_PR) && slbe->Kp)
if ((vcpu->arch.shared->msr & MSR_PR) && slbe->Kp)
key = 4;
else if (!(vcpu->arch.msr & MSR_PR) && slbe->Ks)
else if (!(vcpu->arch.shared->msr & MSR_PR) && slbe->Ks)
key = 4;
for (i=0; i<16; i+=2) {
......@@ -381,7 +397,7 @@ static void kvmppc_mmu_book3s_64_slbia(struct kvm_vcpu *vcpu)
for (i = 1; i < vcpu_book3s->slb_nr; i++)
vcpu_book3s->slb[i].valid = false;
if (vcpu->arch.msr & MSR_IR) {
if (vcpu->arch.shared->msr & MSR_IR) {
kvmppc_mmu_flush_segments(vcpu);
kvmppc_mmu_map_segment(vcpu, kvmppc_get_pc(vcpu));
}
......@@ -445,14 +461,15 @@ static int kvmppc_mmu_book3s_64_esid_to_vsid(struct kvm_vcpu *vcpu, ulong esid,
ulong ea = esid << SID_SHIFT;
struct kvmppc_slb *slb;
u64 gvsid = esid;
ulong mp_ea = vcpu->arch.magic_page_ea;
if (vcpu->arch.msr & (MSR_DR|MSR_IR)) {
if (vcpu->arch.shared->msr & (MSR_DR|MSR_IR)) {
slb = kvmppc_mmu_book3s_64_find_slbe(to_book3s(vcpu), ea);
if (slb)
gvsid = slb->vsid;
}
switch (vcpu->arch.msr & (MSR_DR|MSR_IR)) {
switch (vcpu->arch.shared->msr & (MSR_DR|MSR_IR)) {
case 0:
*vsid = VSID_REAL | esid;
break;
......@@ -464,7 +481,7 @@ static int kvmppc_mmu_book3s_64_esid_to_vsid(struct kvm_vcpu *vcpu, ulong esid,
break;
case MSR_DR|MSR_IR:
if (!slb)
return -ENOENT;
goto no_slb;
*vsid = gvsid;
break;
......@@ -473,10 +490,21 @@ static int kvmppc_mmu_book3s_64_esid_to_vsid(struct kvm_vcpu *vcpu, ulong esid,
break;
}
if (vcpu->arch.msr & MSR_PR)
if (vcpu->arch.shared->msr & MSR_PR)
*vsid |= VSID_PR;
return 0;
no_slb:
/* Catch magic page case */
if (unlikely(mp_ea) &&
unlikely(esid == (mp_ea >> SID_SHIFT)) &&
!(vcpu->arch.shared->msr & MSR_PR)) {
*vsid = VSID_REAL | esid;
return 0;
}
return -EINVAL;
}
static bool kvmppc_mmu_book3s_64_is_dcbz32(struct kvm_vcpu *vcpu)
......
arch/powerpc/kvm/book3s_64_mmu_host.c
View file @ 1765a1fe
......@@ -20,7 +20,6 @@
*/
#include <linux/kvm_host.h>
#include <linux/hash.h>
#include <asm/kvm_ppc.h>
#include <asm/kvm_book3s.h>
......@@ -28,24 +27,9 @@
#include <asm/machdep.h>
#include <asm/mmu_context.h>
#include <asm/hw_irq.h>
#include "trace.h"
#define PTE_SIZE 12
#define VSID_ALL 0
/* #define DEBUG_MMU */
/* #define DEBUG_SLB */
#ifdef DEBUG_MMU
#define dprintk_mmu(a, ...) printk(KERN_INFO a, __VA_ARGS__)
#else
#define dprintk_mmu(a, ...) do { } while(0)
#endif
#ifdef DEBUG_SLB
#define dprintk_slb(a, ...) printk(KERN_INFO a, __VA_ARGS__)
#else
#define dprintk_slb(a, ...) do { } while(0)
#endif
void kvmppc_mmu_invalidate_pte(struct kvm_vcpu *vcpu, struct hpte_cache *pte)
{
......@@ -58,34 +42,39 @@ void kvmppc_mmu_invalidate_pte(struct kvm_vcpu *vcpu, struct hpte_cache *pte)
* a hash, so we don't waste cycles on looping */
static u16 kvmppc_sid_hash(struct kvm_vcpu *vcpu, u64 gvsid)
{
return hash_64(gvsid, SID_MAP_BITS);
return (u16)(((gvsid >> (SID_MAP_BITS * 7)) & SID_MAP_MASK) ^
((gvsid >> (SID_MAP_BITS * 6)) & SID_MAP_MASK) ^
((gvsid >> (SID_MAP_BITS * 5)) & SID_MAP_MASK) ^
((gvsid >> (SID_MAP_BITS * 4)) & SID_MAP_MASK) ^
((gvsid >> (SID_MAP_BITS * 3)) & SID_MAP_MASK) ^
((gvsid >> (SID_MAP_BITS * 2)) & SID_MAP_MASK) ^
((gvsid >> (SID_MAP_BITS * 1)) & SID_MAP_MASK) ^
((gvsid >> (SID_MAP_BITS * 0)) & SID_MAP_MASK));
}
static struct kvmppc_sid_map *find_sid_vsid(struct kvm_vcpu *vcpu, u64 gvsid)
{
struct kvmppc_sid_map *map;
u16 sid_map_mask;
if (vcpu->arch.msr & MSR_PR)
if (vcpu->arch.shared->msr & MSR_PR)
gvsid |= VSID_PR;
sid_map_mask = kvmppc_sid_hash(vcpu, gvsid);
map = &to_book3s(vcpu)->sid_map[sid_map_mask];
if (map->guest_vsid == gvsid) {
dprintk_slb("SLB: Searching: 0x%llx -> 0x%llx\n",
gvsid, map->host_vsid);
if (map->valid && (map->guest_vsid == gvsid)) {
trace_kvm_book3s_slb_found(gvsid, map->host_vsid);
return map;
}
map = &to_book3s(vcpu)->sid_map[SID_MAP_MASK - sid_map_mask];
if (map->guest_vsid == gvsid) {
dprintk_slb("SLB: Searching 0x%llx -> 0x%llx\n",
gvsid, map->host_vsid);
if (map->valid && (map->guest_vsid == gvsid)) {
trace_kvm_book3s_slb_found(gvsid, map->host_vsid);
return map;
}
dprintk_slb("SLB: Searching %d/%d: 0x%llx -> not found\n",
sid_map_mask, SID_MAP_MASK - sid_map_mask, gvsid);
trace_kvm_book3s_slb_fail(sid_map_mask, gvsid);
return NULL;
}
......@@ -101,18 +90,13 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte)
struct kvmppc_sid_map *map;
/* Get host physical address for gpa */
hpaddr = gfn_to_pfn(vcpu->kvm, orig_pte->raddr >> PAGE_SHIFT);
if (kvm_is_error_hva(hpaddr)) {
hpaddr = kvmppc_gfn_to_pfn(vcpu, orig_pte->raddr >> PAGE_SHIFT);
if (is_error_pfn(hpaddr)) {
printk(KERN_INFO "Couldn't get guest page for gfn %lx!\n", orig_pte->eaddr);
return -EINVAL;
}
hpaddr <<= PAGE_SHIFT;
#if PAGE_SHIFT == 12
#elif PAGE_SHIFT == 16
hpaddr |= orig_pte->raddr & 0xf000;
#else
#error Unknown page size
#endif
hpaddr |= orig_pte->raddr & (~0xfffULL & ~PAGE_MASK);
/* and write the mapping ea -> hpa into the pt */
vcpu->arch.mmu.esid_to_vsid(vcpu, orig_pte->eaddr >> SID_SHIFT, &vsid);
......@@ -161,10 +145,7 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte)
} else {
struct hpte_cache *pte = kvmppc_mmu_hpte_cache_next(vcpu);
dprintk_mmu("KVM: %c%c Map 0x%lx: [%lx] 0x%lx (0x%llx) -> %lx\n",
((rflags & HPTE_R_PP) == 3) ? '-' : 'w',
(rflags & HPTE_R_N) ? '-' : 'x',
orig_pte->eaddr, hpteg, va, orig_pte->vpage, hpaddr);
trace_kvm_book3s_64_mmu_map(rflags, hpteg, va, hpaddr, orig_pte);
/* The ppc_md code may give us a secondary entry even though we
asked for a primary. Fix up. */
......@@ -191,7 +172,7 @@ static struct kvmppc_sid_map *create_sid_map(struct kvm_vcpu *vcpu, u64 gvsid)
u16 sid_map_mask;
static int backwards_map = 0;
if (vcpu->arch.msr & MSR_PR)
if (vcpu->arch.shared->msr & MSR_PR)
gvsid |= VSID_PR;
/* We might get collisions that trap in preceding order, so let's
......@@ -219,8 +200,7 @@ static struct kvmppc_sid_map *create_sid_map(struct kvm_vcpu *vcpu, u64 gvsid)
map->guest_vsid = gvsid;
map->valid = true;
dprintk_slb("SLB: New mapping at %d: 0x%llx -> 0x%llx\n",
sid_map_mask, gvsid, map->host_vsid);
trace_kvm_book3s_slb_map(sid_map_mask, gvsid, map->host_vsid);
return map;
}
......@@ -292,7 +272,7 @@ int kvmppc_mmu_map_segment(struct kvm_vcpu *vcpu, ulong eaddr)
to_svcpu(vcpu)->slb[slb_index].esid = slb_esid;
to_svcpu(vcpu)->slb[slb_index].vsid = slb_vsid;
dprintk_slb("slbmte %#llx, %#llx\n", slb_vsid, slb_esid);
trace_kvm_book3s_slbmte(slb_vsid, slb_esid);
return 0;
}
......@@ -306,7 +286,7 @@ void kvmppc_mmu_flush_segments(struct kvm_vcpu *vcpu)
void kvmppc_mmu_destroy(struct kvm_vcpu *vcpu)
{
kvmppc_mmu_hpte_destroy(vcpu);
__destroy_context(to_book3s(vcpu)->context_id);
__destroy_context(to_book3s(vcpu)->context_id[0]);
}
int kvmppc_mmu_init(struct kvm_vcpu *vcpu)
......@@ -317,10 +297,10 @@ int kvmppc_mmu_init(struct kvm_vcpu *vcpu)
err = __init_new_context();
if (err < 0)
return -1;
vcpu3s->context_id = err;
vcpu3s->context_id[0] = err;
vcpu3s->vsid_max = ((vcpu3s->context_id + 1) << USER_ESID_BITS) - 1;
vcpu3s->vsid_first = vcpu3s->context_id << USER_ESID_BITS;
vcpu3s->vsid_max = ((vcpu3s->context_id[0] + 1) << USER_ESID_BITS) - 1;
vcpu3s->vsid_first = vcpu3s->context_id[0] << USER_ESID_BITS;
vcpu3s->vsid_next = vcpu3s->vsid_first;
kvmppc_mmu_hpte_init(vcpu);
......
arch/powerpc/kvm/book3s_emulate.c
View file @ 1765a1fe
......@@ -73,8 +73,8 @@ int kvmppc_core_emulate_op(struct kvm_run *run, struct kvm_vcpu *vcpu,
switch (get_xop(inst)) {
case OP_19_XOP_RFID:
case OP_19_XOP_RFI:
kvmppc_set_pc(vcpu, vcpu->arch.srr0);
kvmppc_set_msr(vcpu, vcpu->arch.srr1);
kvmppc_set_pc(vcpu, vcpu->arch.shared->srr0);
kvmppc_set_msr(vcpu, vcpu->arch.shared->srr1);
*advance = 0;
break;
......@@ -86,14 +86,15 @@ int kvmppc_core_emulate_op(struct kvm_run *run, struct kvm_vcpu *vcpu,
case 31:
switch (get_xop(inst)) {
case OP_31_XOP_MFMSR:
kvmppc_set_gpr(vcpu, get_rt(inst), vcpu->arch.msr);
kvmppc_set_gpr(vcpu, get_rt(inst),
vcpu->arch.shared->msr);
break;
case OP_31_XOP_MTMSRD:
{
ulong rs = kvmppc_get_gpr(vcpu, get_rs(inst));
if (inst & 0x10000) {
vcpu->arch.msr &= ~(MSR_RI | MSR_EE);
vcpu->arch.msr |= rs & (MSR_RI | MSR_EE);
vcpu->arch.shared->msr &= ~(MSR_RI | MSR_EE);
vcpu->arch.shared->msr |= rs & (MSR_RI | MSR_EE);
} else
kvmppc_set_msr(vcpu, rs);
break;
......@@ -204,14 +205,14 @@ int kvmppc_core_emulate_op(struct kvm_run *run, struct kvm_vcpu *vcpu,
ra = kvmppc_get_gpr(vcpu, get_ra(inst));
addr = (ra + rb) & ~31ULL;
if (!(vcpu->arch.msr & MSR_SF))
if (!(vcpu->arch.shared->msr & MSR_SF))
addr &= 0xffffffff;
vaddr = addr;
r = kvmppc_st(vcpu, &addr, 32, zeros, true);
if ((r == -ENOENT) || (r == -EPERM)) {
*advance = 0;
vcpu->arch.dear = vaddr;
vcpu->arch.shared->dar = vaddr;
to_svcpu(vcpu)->fault_dar = vaddr;
dsisr = DSISR_ISSTORE;
......@@ -220,7 +221,7 @@ int kvmppc_core_emulate_op(struct kvm_run *run, struct kvm_vcpu *vcpu,
else if (r == -EPERM)
dsisr |= DSISR_PROTFAULT;
to_book3s(vcpu)->dsisr = dsisr;
vcpu->arch.shared->dsisr = dsisr;
to_svcpu(vcpu)->fault_dsisr = dsisr;
kvmppc_book3s_queue_irqprio(vcpu,
......@@ -263,7 +264,7 @@ void kvmppc_set_bat(struct kvm_vcpu *vcpu, struct kvmppc_bat *bat, bool upper,
}
}
static u32 kvmppc_read_bat(struct kvm_vcpu *vcpu, int sprn)
static struct kvmppc_bat *kvmppc_find_bat(struct kvm_vcpu *vcpu, int sprn)
{
struct kvmppc_vcpu_book3s *vcpu_book3s = to_book3s(vcpu);
struct kvmppc_bat *bat;
......@@ -285,35 +286,7 @@ static u32 kvmppc_read_bat(struct kvm_vcpu *vcpu, int sprn)
BUG();
}
if (sprn % 2)
return bat->raw >> 32;
else
return bat->raw;
}
static void kvmppc_write_bat(struct kvm_vcpu *vcpu, int sprn, u32 val)
{
struct kvmppc_vcpu_book3s *vcpu_book3s = to_book3s(vcpu);
struct kvmppc_bat *bat;
switch (sprn) {
case SPRN_IBAT0U ... SPRN_IBAT3L:
bat = &vcpu_book3s->ibat[(sprn - SPRN_IBAT0U) / 2];
break;
case SPRN_IBAT4U ... SPRN_IBAT7L:
bat = &vcpu_book3s->ibat[4 + ((sprn - SPRN_IBAT4U) / 2)];
break;
case SPRN_DBAT0U ... SPRN_DBAT3L:
bat = &vcpu_book3s->dbat[(sprn - SPRN_DBAT0U) / 2];
break;
case SPRN_DBAT4U ... SPRN_DBAT7L:
bat = &vcpu_book3s->dbat[4 + ((sprn - SPRN_DBAT4U) / 2)];
break;
default:
BUG();
}
kvmppc_set_bat(vcpu, bat, !(sprn % 2), val);
return bat;
}
int kvmppc_core_emulate_mtspr(struct kvm_vcpu *vcpu, int sprn, int rs)
......@@ -326,10 +299,10 @@ int kvmppc_core_emulate_mtspr(struct kvm_vcpu *vcpu, int sprn, int rs)
to_book3s(vcpu)->sdr1 = spr_val;
break;
case SPRN_DSISR:
to_book3s(vcpu)->dsisr = spr_val;
vcpu->arch.shared->dsisr = spr_val;
break;
case SPRN_DAR:
vcpu->arch.dear = spr_val;
vcpu->arch.shared->dar = spr_val;
break;
case SPRN_HIOR:
to_book3s(vcpu)->hior = spr_val;
......@@ -338,12 +311,16 @@ int kvmppc_core_emulate_mtspr(struct kvm_vcpu *vcpu, int sprn, int rs)
case SPRN_IBAT4U ... SPRN_IBAT7L:
case SPRN_DBAT0U ... SPRN_DBAT3L:
case SPRN_DBAT4U ... SPRN_DBAT7L:
kvmppc_write_bat(vcpu, sprn, (u32)spr_val);
{
struct kvmppc_bat *bat = kvmppc_find_bat(vcpu, sprn);
kvmppc_set_bat(vcpu, bat, !(sprn % 2), (u32)spr_val);
/* BAT writes happen so rarely that we're ok to flush
* everything here */
kvmppc_mmu_pte_flush(vcpu, 0, 0);
kvmppc_mmu_flush_segments(vcpu);
break;
}
case SPRN_HID0:
to_book3s(vcpu)->hid[0] = spr_val;
break;
......@@ -433,16 +410,24 @@ int kvmppc_core_emulate_mfspr(struct kvm_vcpu *vcpu, int sprn, int rt)
case SPRN_IBAT4U ... SPRN_IBAT7L:
case SPRN_DBAT0U ... SPRN_DBAT3L:
case SPRN_DBAT4U ... SPRN_DBAT7L:
kvmppc_set_gpr(vcpu, rt, kvmppc_read_bat(vcpu, sprn));
{
struct kvmppc_bat *bat = kvmppc_find_bat(vcpu, sprn);
if (sprn % 2)
kvmppc_set_gpr(vcpu, rt, bat->raw >> 32);
else
kvmppc_set_gpr(vcpu, rt, bat->raw);
break;
}
case SPRN_SDR1:
kvmppc_set_gpr(vcpu, rt, to_book3s(vcpu)->sdr1);
break;
case SPRN_DSISR:
kvmppc_set_gpr(vcpu, rt, to_book3s(vcpu)->dsisr);
kvmppc_set_gpr(vcpu, rt, vcpu->arch.shared->dsisr);
break;
case SPRN_DAR:
kvmppc_set_gpr(vcpu, rt, vcpu->arch.dear);
kvmppc_set_gpr(vcpu, rt, vcpu->arch.shared->dar);
break;
case SPRN_HIOR:
kvmppc_set_gpr(vcpu, rt, to_book3s(vcpu)->hior);
......
arch/powerpc/kvm/book3s_mmu_hpte.c
View file @ 1765a1fe
......@@ -21,6 +21,7 @@
#include <linux/kvm_host.h>
#include <linux/hash.h>
#include <linux/slab.h>
#include "trace.h"
#include <asm/kvm_ppc.h>
#include <asm/kvm_book3s.h>
......@@ -30,14 +31,6 @@
#define PTE_SIZE 12
/* #define DEBUG_MMU */
#ifdef DEBUG_MMU
#define dprintk_mmu(a, ...) printk(KERN_INFO a, __VA_ARGS__)
#else
#define dprintk_mmu(a, ...) do { } while(0)
#endif
static struct kmem_cache *hpte_cache;
static inline u64 kvmppc_mmu_hash_pte(u64 eaddr)
......@@ -45,6 +38,12 @@ static inline u64 kvmppc_mmu_hash_pte(u64 eaddr)
return hash_64(eaddr >> PTE_SIZE, HPTEG_HASH_BITS_PTE);
}
static inline u64 kvmppc_mmu_hash_pte_long(u64 eaddr)
{
return hash_64((eaddr & 0x0ffff000) >> PTE_SIZE,
HPTEG_HASH_BITS_PTE_LONG);
}
static inline u64 kvmppc_mmu_hash_vpte(u64 vpage)
{
return hash_64(vpage & 0xfffffffffULL, HPTEG_HASH_BITS_VPTE);
......@@ -60,77 +59,128 @@ void kvmppc_mmu_hpte_cache_map(struct kvm_vcpu *vcpu, struct hpte_cache *pte)
{
u64 index;
trace_kvm_book3s_mmu_map(pte);
spin_lock(&vcpu->arch.mmu_lock);
/* Add to ePTE list */
index = kvmppc_mmu_hash_pte(pte->pte.eaddr);
hlist_add_head(&pte->list_pte, &vcpu->arch.hpte_hash_pte[index]);
hlist_add_head_rcu(&pte->list_pte, &vcpu->arch.hpte_hash_pte[index]);
/* Add to ePTE_long list */
index = kvmppc_mmu_hash_pte_long(pte->pte.eaddr);
hlist_add_head_rcu(&pte->list_pte_long,
&vcpu->arch.hpte_hash_pte_long[index]);
/* Add to vPTE list */
index = kvmppc_mmu_hash_vpte(pte->pte.vpage);
hlist_add_head(&pte->list_vpte, &vcpu->arch.hpte_hash_vpte[index]);
hlist_add_head_rcu(&pte->list_vpte, &vcpu->arch.hpte_hash_vpte[index]);
/* Add to vPTE_long list */
index = kvmppc_mmu_hash_vpte_long(pte->pte.vpage);
hlist_add_head(&pte->list_vpte_long,
hlist_add_head_rcu(&pte->list_vpte_long,
&vcpu->arch.hpte_hash_vpte_long[index]);
spin_unlock(&vcpu->arch.mmu_lock);
}
static void free_pte_rcu(struct rcu_head *head)
{
struct hpte_cache *pte = container_of(head, struct hpte_cache, rcu_head);
kmem_cache_free(hpte_cache, pte);
}
static void invalidate_pte(struct kvm_vcpu *vcpu, struct hpte_cache *pte)
{
dprintk_mmu("KVM: Flushing SPT: 0x%lx (0x%llx) -> 0x%llx\n",
pte->pte.eaddr, pte->pte.vpage, pte->host_va);
trace_kvm_book3s_mmu_invalidate(pte);
/* Different for 32 and 64 bit */
kvmppc_mmu_invalidate_pte(vcpu, pte);
spin_lock(&vcpu->arch.mmu_lock);
/* pte already invalidated in between? */
if (hlist_unhashed(&pte->list_pte)) {
spin_unlock(&vcpu->arch.mmu_lock);
return;
}
hlist_del_init_rcu(&pte->list_pte);
hlist_del_init_rcu(&pte->list_pte_long);
hlist_del_init_rcu(&pte->list_vpte);
hlist_del_init_rcu(&pte->list_vpte_long);
if (pte->pte.may_write)
kvm_release_pfn_dirty(pte->pfn);
else
kvm_release_pfn_clean(pte->pfn);
hlist_del(&pte->list_pte);
hlist_del(&pte->list_vpte);
hlist_del(&pte->list_vpte_long);
spin_unlock(&vcpu->arch.mmu_lock);
vcpu->arch.hpte_cache_count--;
kmem_cache_free(hpte_cache, pte);
call_rcu(&pte->rcu_head, free_pte_rcu);
}
static void kvmppc_mmu_pte_flush_all(struct kvm_vcpu *vcpu)
{
struct hpte_cache *pte;
struct hlist_node *node, *tmp;
struct hlist_node *node;
int i;
rcu_read_lock();
for (i = 0; i < HPTEG_HASH_NUM_VPTE_LONG; i++) {
struct hlist_head *list = &vcpu->arch.hpte_hash_vpte_long[i];
hlist_for_each_entry_safe(pte, node, tmp, list, list_vpte_long)
hlist_for_each_entry_rcu(pte, node, list, list_vpte_long)
invalidate_pte(vcpu, pte);
}
rcu_read_unlock();
}
static void kvmppc_mmu_pte_flush_page(struct kvm_vcpu *vcpu, ulong guest_ea)
{
struct hlist_head *list;
struct hlist_node *node, *tmp;
struct hlist_node *node;
struct hpte_cache *pte;
/* Find the list of entries in the map */
list = &vcpu->arch.hpte_hash_pte[kvmppc_mmu_hash_pte(guest_ea)];
rcu_read_lock();
/* Check the list for matching entries and invalidate */
hlist_for_each_entry_safe(pte, node, tmp, list, list_pte)
hlist_for_each_entry_rcu(pte, node, list, list_pte)
if ((pte->pte.eaddr & ~0xfffUL) == guest_ea)
invalidate_pte(vcpu, pte);
rcu_read_unlock();
}
void kvmppc_mmu_pte_flush(struct kvm_vcpu *vcpu, ulong guest_ea, ulong ea_mask)
static void kvmppc_mmu_pte_flush_long(struct kvm_vcpu *vcpu, ulong guest_ea)
{
u64 i;
struct hlist_head *list;
struct hlist_node *node;
struct hpte_cache *pte;
dprintk_mmu("KVM: Flushing %d Shadow PTEs: 0x%lx & 0x%lx\n",
vcpu->arch.hpte_cache_count, guest_ea, ea_mask);
/* Find the list of entries in the map */
list = &vcpu->arch.hpte_hash_pte_long[
kvmppc_mmu_hash_pte_long(guest_ea)];
rcu_read_lock();
/* Check the list for matching entries and invalidate */
hlist_for_each_entry_rcu(pte, node, list, list_pte_long)
if ((pte->pte.eaddr & 0x0ffff000UL) == guest_ea)
invalidate_pte(vcpu, pte);
rcu_read_unlock();
}
void kvmppc_mmu_pte_flush(struct kvm_vcpu *vcpu, ulong guest_ea, ulong ea_mask)
{
trace_kvm_book3s_mmu_flush("", vcpu, guest_ea, ea_mask);
guest_ea &= ea_mask;
switch (ea_mask) {
......@@ -138,9 +188,7 @@ void kvmppc_mmu_pte_flush(struct kvm_vcpu *vcpu, ulong guest_ea, ulong ea_mask)
kvmppc_mmu_pte_flush_page(vcpu, guest_ea);
break;
case 0x0ffff000:
/* 32-bit flush w/o segment, go through all possible segments */
for (i = 0; i < 0x100000000ULL; i += 0x10000000ULL)
kvmppc_mmu_pte_flush(vcpu, guest_ea | i, ~0xfffUL);
kvmppc_mmu_pte_flush_long(vcpu, guest_ea);
break;
case 0:
/* Doing a complete flush -> start from scratch */
......@@ -156,39 +204,46 @@ void kvmppc_mmu_pte_flush(struct kvm_vcpu *vcpu, ulong guest_ea, ulong ea_mask)
static void kvmppc_mmu_pte_vflush_short(struct kvm_vcpu *vcpu, u64 guest_vp)
{
struct hlist_head *list;
struct hlist_node *node, *tmp;
struct hlist_node *node;
struct hpte_cache *pte;
u64 vp_mask = 0xfffffffffULL;
list = &vcpu->arch.hpte_hash_vpte[kvmppc_mmu_hash_vpte(guest_vp)];
rcu_read_lock();
/* Check the list for matching entries and invalidate */
hlist_for_each_entry_safe(pte, node, tmp, list, list_vpte)
hlist_for_each_entry_rcu(pte, node, list, list_vpte)
if ((pte->pte.vpage & vp_mask) == guest_vp)
invalidate_pte(vcpu, pte);
rcu_read_unlock();
}
/* Flush with mask 0xffffff000 */
static void kvmppc_mmu_pte_vflush_long(struct kvm_vcpu *vcpu, u64 guest_vp)
{
struct hlist_head *list;
struct hlist_node *node, *tmp;
struct hlist_node *node;
struct hpte_cache *pte;
u64 vp_mask = 0xffffff000ULL;
list = &vcpu->arch.hpte_hash_vpte_long[
kvmppc_mmu_hash_vpte_long(guest_vp)];
rcu_read_lock();
/* Check the list for matching entries and invalidate */
hlist_for_each_entry_safe(pte, node, tmp, list, list_vpte_long)
hlist_for_each_entry_rcu(pte, node, list, list_vpte_long)
if ((pte->pte.vpage & vp_mask) == guest_vp)
invalidate_pte(vcpu, pte);
rcu_read_unlock();
}
void kvmppc_mmu_pte_vflush(struct kvm_vcpu *vcpu, u64 guest_vp, u64 vp_mask)
{
dprintk_mmu("KVM: Flushing %d Shadow vPTEs: 0x%llx & 0x%llx\n",
vcpu->arch.hpte_cache_count, guest_vp, vp_mask);
trace_kvm_book3s_mmu_flush("v", vcpu, guest_vp, vp_mask);
guest_vp &= vp_mask;
switch(vp_mask) {
......@@ -206,21 +261,24 @@ void kvmppc_mmu_pte_vflush(struct kvm_vcpu *vcpu, u64 guest_vp, u64 vp_mask)
void kvmppc_mmu_pte_pflush(struct kvm_vcpu *vcpu, ulong pa_start, ulong pa_end)
{
struct hlist_node *node, *tmp;
struct hlist_node *node;
struct hpte_cache *pte;
int i;
dprintk_mmu("KVM: Flushing %d Shadow pPTEs: 0x%lx - 0x%lx\n",
vcpu->arch.hpte_cache_count, pa_start, pa_end);
trace_kvm_book3s_mmu_flush("p", vcpu, pa_start, pa_end);
rcu_read_lock();
for (i = 0; i < HPTEG_HASH_NUM_VPTE_LONG; i++) {
struct hlist_head *list = &vcpu->arch.hpte_hash_vpte_long[i];
hlist_for_each_entry_safe(pte, node, tmp, list, list_vpte_long)
hlist_for_each_entry_rcu(pte, node, list, list_vpte_long)
if ((pte->pte.raddr >= pa_start) &&
(pte->pte.raddr < pa_end))
invalidate_pte(vcpu, pte);
}
rcu_read_unlock();
}
struct hpte_cache *kvmppc_mmu_hpte_cache_next(struct kvm_vcpu *vcpu)
......@@ -254,11 +312,15 @@ int kvmppc_mmu_hpte_init(struct kvm_vcpu *vcpu)
/* init hpte lookup hashes */
kvmppc_mmu_hpte_init_hash(vcpu->arch.hpte_hash_pte,
ARRAY_SIZE(vcpu->arch.hpte_hash_pte));
kvmppc_mmu_hpte_init_hash(vcpu->arch.hpte_hash_pte_long,
ARRAY_SIZE(vcpu->arch.hpte_hash_pte_long));
kvmppc_mmu_hpte_init_hash(vcpu->arch.hpte_hash_vpte,
ARRAY_SIZE(vcpu->arch.hpte_hash_vpte));
kvmppc_mmu_hpte_init_hash(vcpu->arch.hpte_hash_vpte_long,
ARRAY_SIZE(vcpu->arch.hpte_hash_vpte_long));
spin_lock_init(&vcpu->arch.mmu_lock);
return 0;
}
......
arch/powerpc/kvm/book3s_paired_singles.c
View file @ 1765a1fe
......@@ -165,14 +165,15 @@ static inline void kvmppc_sync_qpr(struct kvm_vcpu *vcpu, int rt)
static void kvmppc_inject_pf(struct kvm_vcpu *vcpu, ulong eaddr, bool is_store)
{
u64 dsisr;
struct kvm_vcpu_arch_shared *shared = vcpu->arch.shared;
vcpu->arch.msr = kvmppc_set_field(vcpu->arch.msr, 33, 36, 0);
vcpu->arch.msr = kvmppc_set_field(vcpu->arch.msr, 42, 47, 0);
vcpu->arch.dear = eaddr;
shared->msr = kvmppc_set_field(shared->msr, 33, 36, 0);
shared->msr = kvmppc_set_field(shared->msr, 42, 47, 0);
shared->dar = eaddr;
/* Page Fault */
dsisr = kvmppc_set_field(0, 33, 33, 1);
if (is_store)
to_book3s(vcpu)->dsisr = kvmppc_set_field(dsisr, 38, 38, 1);
shared->dsisr = kvmppc_set_field(dsisr, 38, 38, 1);
kvmppc_book3s_queue_irqprio(vcpu, BOOK3S_INTERRUPT_DATA_STORAGE);
}
......@@ -658,7 +659,7 @@ int kvmppc_emulate_paired_single(struct kvm_run *run, struct kvm_vcpu *vcpu)
if (!kvmppc_inst_is_paired_single(vcpu, inst))
return EMULATE_FAIL;
if (!(vcpu->arch.msr & MSR_FP)) {
if (!(vcpu->arch.shared->msr & MSR_FP)) {
kvmppc_book3s_queue_irqprio(vcpu, BOOK3S_INTERRUPT_FP_UNAVAIL);
return EMULATE_AGAIN;
}
......
arch/powerpc/kvm/book3s_rmhandlers.S
View file @ 1765a1fe
......@@ -202,8 +202,25 @@ _GLOBAL(kvmppc_rmcall)
#if defined(CONFIG_PPC_BOOK3S_32)
#define STACK_LR INT_FRAME_SIZE+4
/* load_up_xxx have to run with MSR_DR=0 on Book3S_32 */
#define MSR_EXT_START \
PPC_STL r20, _NIP(r1); \
mfmsr r20; \
LOAD_REG_IMMEDIATE(r3, MSR_DR|MSR_EE); \
andc r3,r20,r3; /* Disable DR,EE */ \
mtmsr r3; \
sync
#define MSR_EXT_END \
mtmsr r20; /* Enable DR,EE */ \
sync; \
PPC_LL r20, _NIP(r1)
#elif defined(CONFIG_PPC_BOOK3S_64)
#define STACK_LR _LINK
#define MSR_EXT_START
#define MSR_EXT_END
#endif
/*
......@@ -215,19 +232,12 @@ _GLOBAL(kvmppc_load_up_ ## what); \
PPC_STLU r1, -INT_FRAME_SIZE(r1); \
mflr r3; \
PPC_STL r3, STACK_LR(r1); \
PPC_STL r20, _NIP(r1); \
mfmsr r20; \
LOAD_REG_IMMEDIATE(r3, MSR_DR|MSR_EE); \
andc r3,r20,r3; /* Disable DR,EE */ \
mtmsr r3; \
sync; \
MSR_EXT_START; \
\
bl FUNC(load_up_ ## what); \
\
mtmsr r20; /* Enable DR,EE */ \
sync; \
MSR_EXT_END; \
PPC_LL r3, STACK_LR(r1); \
PPC_LL r20, _NIP(r1); \
mtlr r3; \
addi r1, r1, INT_FRAME_SIZE; \
blr
......@@ -242,10 +252,10 @@ define_load_up(vsx)
.global kvmppc_trampoline_lowmem
kvmppc_trampoline_lowmem:
.long kvmppc_handler_lowmem_trampoline - CONFIG_KERNEL_START
PPC_LONG kvmppc_handler_lowmem_trampoline - CONFIG_KERNEL_START
.global kvmppc_trampoline_enter
kvmppc_trampoline_enter:
.long kvmppc_handler_trampoline_enter - CONFIG_KERNEL_START
PPC_LONG kvmppc_handler_trampoline_enter - CONFIG_KERNEL_START
#include "book3s_segment.S"
arch/powerpc/kvm/booke.c
View file @ 1765a1fe
......@@ -62,9 +62,10 @@ void kvmppc_dump_vcpu(struct kvm_vcpu *vcpu)
{
int i;
printk("pc: %08lx msr: %08lx\n", vcpu->arch.pc, vcpu->arch.msr);
printk("pc: %08lx msr: %08llx\n", vcpu->arch.pc, vcpu->arch.shared->msr);
printk("lr: %08lx ctr: %08lx\n", vcpu->arch.lr, vcpu->arch.ctr);
printk("srr0: %08lx srr1: %08lx\n", vcpu->arch.srr0, vcpu->arch.srr1);
printk("srr0: %08llx srr1: %08llx\n", vcpu->arch.shared->srr0,
vcpu->arch.shared->srr1);
printk("exceptions: %08lx\n", vcpu->arch.pending_exceptions);
......@@ -130,13 +131,19 @@ void kvmppc_core_dequeue_dec(struct kvm_vcpu *vcpu)
void kvmppc_core_queue_external(struct kvm_vcpu *vcpu,
struct kvm_interrupt *irq)
{
kvmppc_booke_queue_irqprio(vcpu, BOOKE_IRQPRIO_EXTERNAL);
unsigned int prio = BOOKE_IRQPRIO_EXTERNAL;
if (irq->irq == KVM_INTERRUPT_SET_LEVEL)
prio = BOOKE_IRQPRIO_EXTERNAL_LEVEL;
kvmppc_booke_queue_irqprio(vcpu, prio);
}
void kvmppc_core_dequeue_external(struct kvm_vcpu *vcpu,
struct kvm_interrupt *irq)
{
clear_bit(BOOKE_IRQPRIO_EXTERNAL, &vcpu->arch.pending_exceptions);
clear_bit(BOOKE_IRQPRIO_EXTERNAL_LEVEL, &vcpu->arch.pending_exceptions);
}
/* Deliver the interrupt of the corresponding priority, if possible. */
......@@ -146,6 +153,26 @@ static int kvmppc_booke_irqprio_deliver(struct kvm_vcpu *vcpu,
int allowed = 0;
ulong uninitialized_var(msr_mask);
bool update_esr = false, update_dear = false;
ulong crit_raw = vcpu->arch.shared->critical;
ulong crit_r1 = kvmppc_get_gpr(vcpu, 1);
bool crit;
bool keep_irq = false;
/* Truncate crit indicators in 32 bit mode */
if (!(vcpu->arch.shared->msr & MSR_SF)) {
crit_raw &= 0xffffffff;
crit_r1 &= 0xffffffff;
}
/* Critical section when crit == r1 */
crit = (crit_raw == crit_r1);
/* ... and we're in supervisor mode */
crit = crit && !(vcpu->arch.shared->msr & MSR_PR);
if (priority == BOOKE_IRQPRIO_EXTERNAL_LEVEL) {
priority = BOOKE_IRQPRIO_EXTERNAL;
keep_irq = true;
}
switch (priority) {
case BOOKE_IRQPRIO_DTLB_MISS:
......@@ -169,35 +196,37 @@ static int kvmppc_booke_irqprio_deliver(struct kvm_vcpu *vcpu,
break;
case BOOKE_IRQPRIO_CRITICAL:
case BOOKE_IRQPRIO_WATCHDOG:
allowed = vcpu->arch.msr & MSR_CE;
allowed = vcpu->arch.shared->msr & MSR_CE;
msr_mask = MSR_ME;
break;
case BOOKE_IRQPRIO_MACHINE_CHECK:
allowed = vcpu->arch.msr & MSR_ME;
allowed = vcpu->arch.shared->msr & MSR_ME;
msr_mask = 0;
break;
case BOOKE_IRQPRIO_EXTERNAL:
case BOOKE_IRQPRIO_DECREMENTER:
case BOOKE_IRQPRIO_FIT:
allowed = vcpu->arch.msr & MSR_EE;
allowed = vcpu->arch.shared->msr & MSR_EE;
allowed = allowed && !crit;
msr_mask = MSR_CE|MSR_ME|MSR_DE;
break;
case BOOKE_IRQPRIO_DEBUG:
allowed = vcpu->arch.msr & MSR_DE;
allowed = vcpu->arch.shared->msr & MSR_DE;
msr_mask = MSR_ME;
break;
}
if (allowed) {
vcpu->arch.srr0 = vcpu->arch.pc;
vcpu->arch.srr1 = vcpu->arch.msr;
vcpu->arch.shared->srr0 = vcpu->arch.pc;
vcpu->arch.shared->srr1 = vcpu->arch.shared->msr;
vcpu->arch.pc = vcpu->arch.ivpr | vcpu->arch.ivor[priority];
if (update_esr == true)
vcpu->arch.esr = vcpu->arch.queued_esr;
if (update_dear == true)
vcpu->arch.dear = vcpu->arch.queued_dear;
kvmppc_set_msr(vcpu, vcpu->arch.msr & msr_mask);
vcpu->arch.shared->dar = vcpu->arch.queued_dear;
kvmppc_set_msr(vcpu, vcpu->arch.shared->msr & msr_mask);
if (!keep_irq)
clear_bit(priority, &vcpu->arch.pending_exceptions);
}
......@@ -208,6 +237,7 @@ static int kvmppc_booke_irqprio_deliver(struct kvm_vcpu *vcpu,
void kvmppc_core_deliver_interrupts(struct kvm_vcpu *vcpu)
{
unsigned long *pending = &vcpu->arch.pending_exceptions;
unsigned long old_pending = vcpu->arch.pending_exceptions;
unsigned int priority;
priority = __ffs(*pending);
......@@ -219,6 +249,12 @@ void kvmppc_core_deliver_interrupts(struct kvm_vcpu *vcpu)
BITS_PER_BYTE * sizeof(*pending),
priority + 1);
}
/* Tell the guest about our interrupt status */
if (*pending)
vcpu->arch.shared->int_pending = 1;
else if (old_pending)
vcpu->arch.shared->int_pending = 0;
}
/**
......@@ -265,7 +301,7 @@ int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu,
break;
case BOOKE_INTERRUPT_PROGRAM:
if (vcpu->arch.msr & MSR_PR) {
if (vcpu->arch.shared->msr & MSR_PR) {
/* Program traps generated by user-level software must be handled
* by the guest kernel. */
kvmppc_core_queue_program(vcpu, vcpu->arch.fault_esr);
......@@ -337,7 +373,15 @@ int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu,
break;
case BOOKE_INTERRUPT_SYSCALL:
if (!(vcpu->arch.shared->msr & MSR_PR) &&
(((u32)kvmppc_get_gpr(vcpu, 0)) == KVM_SC_MAGIC_R0)) {
/* KVM PV hypercalls */
kvmppc_set_gpr(vcpu, 3, kvmppc_kvm_pv(vcpu));
r = RESUME_GUEST;
} else {
/* Guest syscalls */
kvmppc_booke_queue_irqprio(vcpu, BOOKE_IRQPRIO_SYSCALL);
}
kvmppc_account_exit(vcpu, SYSCALL_EXITS);
r = RESUME_GUEST;
break;
......@@ -466,15 +510,19 @@ int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu,
/* Initial guest state: 16MB mapping 0 -> 0, PC = 0, MSR = 0, R1 = 16MB */
int kvm_arch_vcpu_setup(struct kvm_vcpu *vcpu)
{
int i;
vcpu->arch.pc = 0;
vcpu->arch.msr = 0;
vcpu->arch.shared->msr = 0;
kvmppc_set_gpr(vcpu, 1, (16<<20) - 8); /* -8 for the callee-save LR slot */
vcpu->arch.shadow_pid = 1;
/* Eye-catching number so we know if the guest takes an interrupt
* before it's programmed its own IVPR. */
/* Eye-catching numbers so we know if the guest takes an interrupt
* before it's programmed its own IVPR/IVORs. */
vcpu->arch.ivpr = 0x55550000;
for (i = 0; i < BOOKE_IRQPRIO_MAX; i++)
vcpu->arch.ivor[i] = 0x7700 | i * 4;
kvmppc_init_timing_stats(vcpu);
......@@ -490,14 +538,14 @@ int kvm_arch_vcpu_ioctl_get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
regs->ctr = vcpu->arch.ctr;
regs->lr = vcpu->arch.lr;
regs->xer = kvmppc_get_xer(vcpu);
regs->msr = vcpu->arch.msr;
regs->srr0 = vcpu->arch.srr0;
regs->srr1 = vcpu->arch.srr1;
regs->msr = vcpu->arch.shared->msr;
regs->srr0 = vcpu->arch.shared->srr0;
regs->srr1 = vcpu->arch.shared->srr1;
regs->pid = vcpu->arch.pid;
regs->sprg0 = vcpu->arch.sprg0;
regs->sprg1 = vcpu->arch.sprg1;
regs->sprg2 = vcpu->arch.sprg2;
regs->sprg3 = vcpu->arch.sprg3;
regs->sprg0 = vcpu->arch.shared->sprg0;
regs->sprg1 = vcpu->arch.shared->sprg1;
regs->sprg2 = vcpu->arch.shared->sprg2;
regs->sprg3 = vcpu->arch.shared->sprg3;
regs->sprg5 = vcpu->arch.sprg4;
regs->sprg6 = vcpu->arch.sprg5;
regs->sprg7 = vcpu->arch.sprg6;
......@@ -518,12 +566,12 @@ int kvm_arch_vcpu_ioctl_set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
vcpu->arch.lr = regs->lr;
kvmppc_set_xer(vcpu, regs->xer);
kvmppc_set_msr(vcpu, regs->msr);
vcpu->arch.srr0 = regs->srr0;
vcpu->arch.srr1 = regs->srr1;
vcpu->arch.sprg0 = regs->sprg0;
vcpu->arch.sprg1 = regs->sprg1;
vcpu->arch.sprg2 = regs->sprg2;
vcpu->arch.sprg3 = regs->sprg3;
vcpu->arch.shared->srr0 = regs->srr0;
vcpu->arch.shared->srr1 = regs->srr1;
vcpu->arch.shared->sprg0 = regs->sprg0;
vcpu->arch.shared->sprg1 = regs->sprg1;
vcpu->arch.shared->sprg2 = regs->sprg2;
vcpu->arch.shared->sprg3 = regs->sprg3;
vcpu->arch.sprg5 = regs->sprg4;
vcpu->arch.sprg6 = regs->sprg5;
vcpu->arch.sprg7 = regs->sprg6;
......
arch/powerpc/kvm/booke.h
View file @ 1765a1fe
......@@ -46,7 +46,9 @@
#define BOOKE_IRQPRIO_FIT 17
#define BOOKE_IRQPRIO_DECREMENTER 18
#define BOOKE_IRQPRIO_PERFORMANCE_MONITOR 19
#define BOOKE_IRQPRIO_MAX 19
/* Internal pseudo-irqprio for level triggered externals */
#define BOOKE_IRQPRIO_EXTERNAL_LEVEL 20
#define BOOKE_IRQPRIO_MAX 20
extern unsigned long kvmppc_booke_handlers;
......@@ -54,12 +56,12 @@ extern unsigned long kvmppc_booke_handlers;
* changing. */
static inline void kvmppc_set_msr(struct kvm_vcpu *vcpu, u32 new_msr)
{
if ((new_msr & MSR_PR) != (vcpu->arch.msr & MSR_PR))
if ((new_msr & MSR_PR) != (vcpu->arch.shared->msr & MSR_PR))
kvmppc_mmu_priv_switch(vcpu, new_msr & MSR_PR);
vcpu->arch.msr = new_msr;
vcpu->arch.shared->msr = new_msr;
if (vcpu->arch.msr & MSR_WE) {
if (vcpu->arch.shared->msr & MSR_WE) {
kvm_vcpu_block(vcpu);
kvmppc_set_exit_type(vcpu, EMULATED_MTMSRWE_EXITS);
};
......
arch/powerpc/kvm/booke_emulate.c
View file @ 1765a1fe
......@@ -31,8 +31,8 @@
static void kvmppc_emul_rfi(struct kvm_vcpu *vcpu)
{
vcpu->arch.pc = vcpu->arch.srr0;
kvmppc_set_msr(vcpu, vcpu->arch.srr1);
vcpu->arch.pc = vcpu->arch.shared->srr0;
kvmppc_set_msr(vcpu, vcpu->arch.shared->srr1);
}
int kvmppc_booke_emulate_op(struct kvm_run *run, struct kvm_vcpu *vcpu,
......@@ -62,7 +62,7 @@ int kvmppc_booke_emulate_op(struct kvm_run *run, struct kvm_vcpu *vcpu,
case OP_31_XOP_MFMSR:
rt = get_rt(inst);
kvmppc_set_gpr(vcpu, rt, vcpu->arch.msr);
kvmppc_set_gpr(vcpu, rt, vcpu->arch.shared->msr);
kvmppc_set_exit_type(vcpu, EMULATED_MFMSR_EXITS);
break;
......@@ -74,13 +74,13 @@ int kvmppc_booke_emulate_op(struct kvm_run *run, struct kvm_vcpu *vcpu,
case OP_31_XOP_WRTEE:
rs = get_rs(inst);
vcpu->arch.msr = (vcpu->arch.msr & ~MSR_EE)
vcpu->arch.shared->msr = (vcpu->arch.shared->msr & ~MSR_EE)
| (kvmppc_get_gpr(vcpu, rs) & MSR_EE);
kvmppc_set_exit_type(vcpu, EMULATED_WRTEE_EXITS);
break;
case OP_31_XOP_WRTEEI:
vcpu->arch.msr = (vcpu->arch.msr & ~MSR_EE)
vcpu->arch.shared->msr = (vcpu->arch.shared->msr & ~MSR_EE)
| (inst & MSR_EE);
kvmppc_set_exit_type(vcpu, EMULATED_WRTEE_EXITS);
break;
......@@ -105,7 +105,7 @@ int kvmppc_booke_emulate_mtspr(struct kvm_vcpu *vcpu, int sprn, int rs)
switch (sprn) {
case SPRN_DEAR:
vcpu->arch.dear = spr_val; break;
vcpu->arch.shared->dar = spr_val; break;
case SPRN_ESR:
vcpu->arch.esr = spr_val; break;
case SPRN_DBCR0:
......@@ -200,7 +200,7 @@ int kvmppc_booke_emulate_mfspr(struct kvm_vcpu *vcpu, int sprn, int rt)
case SPRN_IVPR:
kvmppc_set_gpr(vcpu, rt, vcpu->arch.ivpr); break;
case SPRN_DEAR:
kvmppc_set_gpr(vcpu, rt, vcpu->arch.dear); break;
kvmppc_set_gpr(vcpu, rt, vcpu->arch.shared->dar); break;
case SPRN_ESR:
kvmppc_set_gpr(vcpu, rt, vcpu->arch.esr); break;
case SPRN_DBCR0:
......
arch/powerpc/kvm/booke_interrupts.S
View file @ 1765a1fe
......@@ -415,7 +415,8 @@ lightweight_exit:
lwz r8, VCPU_GPR(r8)(r4)
lwz r3, VCPU_PC(r4)
mtsrr0 r3
lwz r3, VCPU_MSR(r4)
lwz r3, VCPU_SHARED(r4)
lwz r3, VCPU_SHARED_MSR(r3)
oris r3, r3, KVMPPC_MSR_MASK@h
ori r3, r3, KVMPPC_MSR_MASK@l
mtsrr1 r3
......
arch/powerpc/kvm/e500.c
View file @ 1765a1fe
......@@ -117,8 +117,14 @@ struct kvm_vcpu *kvmppc_core_vcpu_create(struct kvm *kvm, unsigned int id)
if (err)
goto uninit_vcpu;
vcpu->arch.shared = (void*)__get_free_page(GFP_KERNEL|__GFP_ZERO);
if (!vcpu->arch.shared)
goto uninit_tlb;
return vcpu;
uninit_tlb:
kvmppc_e500_tlb_uninit(vcpu_e500);
uninit_vcpu:
kvm_vcpu_uninit(vcpu);
free_vcpu:
......@@ -131,6 +137,7 @@ void kvmppc_core_vcpu_free(struct kvm_vcpu *vcpu)
{
struct kvmppc_vcpu_e500 *vcpu_e500 = to_e500(vcpu);
free_page((unsigned long)vcpu->arch.shared);
kvmppc_e500_tlb_uninit(vcpu_e500);
kvm_vcpu_uninit(vcpu);
kmem_cache_free(kvm_vcpu_cache, vcpu_e500);
......
arch/powerpc/kvm/e500_tlb.c
View file @ 1765a1fe
......@@ -226,8 +226,7 @@ static void kvmppc_e500_stlbe_invalidate(struct kvmppc_vcpu_e500 *vcpu_e500,
kvmppc_e500_shadow_release(vcpu_e500, tlbsel, esel);
stlbe->mas1 = 0;
trace_kvm_stlb_inval(index_of(tlbsel, esel), stlbe->mas1, stlbe->mas2,
stlbe->mas3, stlbe->mas7);
trace_kvm_stlb_inval(index_of(tlbsel, esel));
}
static void kvmppc_e500_tlb1_invalidate(struct kvmppc_vcpu_e500 *vcpu_e500,
......@@ -298,7 +297,8 @@ static inline void kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500,
/* Get reference to new page. */
new_page = gfn_to_page(vcpu_e500->vcpu.kvm, gfn);
if (is_error_page(new_page)) {
printk(KERN_ERR "Couldn't get guest page for gfn %lx!\n", gfn);
printk(KERN_ERR "Couldn't get guest page for gfn %lx!\n",
(long)gfn);
kvm_release_page_clean(new_page);
return;
}
......@@ -314,10 +314,10 @@ static inline void kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500,
| MAS1_TID(get_tlb_tid(gtlbe)) | MAS1_TS | MAS1_VALID;
stlbe->mas2 = (gvaddr & MAS2_EPN)
| e500_shadow_mas2_attrib(gtlbe->mas2,
vcpu_e500->vcpu.arch.msr & MSR_PR);
vcpu_e500->vcpu.arch.shared->msr & MSR_PR);
stlbe->mas3 = (hpaddr & MAS3_RPN)
| e500_shadow_mas3_attrib(gtlbe->mas3,
vcpu_e500->vcpu.arch.msr & MSR_PR);
vcpu_e500->vcpu.arch.shared->msr & MSR_PR);
stlbe->mas7 = (hpaddr >> 32) & MAS7_RPN;
trace_kvm_stlb_write(index_of(tlbsel, esel), stlbe->mas1, stlbe->mas2,
......@@ -576,28 +576,28 @@ int kvmppc_e500_emul_tlbwe(struct kvm_vcpu *vcpu)
int kvmppc_mmu_itlb_index(struct kvm_vcpu *vcpu, gva_t eaddr)
{
unsigned int as = !!(vcpu->arch.msr & MSR_IS);
unsigned int as = !!(vcpu->arch.shared->msr & MSR_IS);
return kvmppc_e500_tlb_search(vcpu, eaddr, get_cur_pid(vcpu), as);
}
int kvmppc_mmu_dtlb_index(struct kvm_vcpu *vcpu, gva_t eaddr)
{
unsigned int as = !!(vcpu->arch.msr & MSR_DS);
unsigned int as = !!(vcpu->arch.shared->msr & MSR_DS);
return kvmppc_e500_tlb_search(vcpu, eaddr, get_cur_pid(vcpu), as);
}
void kvmppc_mmu_itlb_miss(struct kvm_vcpu *vcpu)
{
unsigned int as = !!(vcpu->arch.msr & MSR_IS);
unsigned int as = !!(vcpu->arch.shared->msr & MSR_IS);
kvmppc_e500_deliver_tlb_miss(vcpu, vcpu->arch.pc, as);
}
void kvmppc_mmu_dtlb_miss(struct kvm_vcpu *vcpu)
{
unsigned int as = !!(vcpu->arch.msr & MSR_DS);
unsigned int as = !!(vcpu->arch.shared->msr & MSR_DS);
kvmppc_e500_deliver_tlb_miss(vcpu, vcpu->arch.fault_dear, as);
}
......
arch/powerpc/kvm/e500_tlb.h
View file @ 1765a1fe
......@@ -171,7 +171,7 @@ static inline int tlbe_is_host_safe(const struct kvm_vcpu *vcpu,
/* Does it match current guest AS? */
/* XXX what about IS != DS? */
if (get_tlb_ts(tlbe) != !!(vcpu->arch.msr & MSR_IS))
if (get_tlb_ts(tlbe) != !!(vcpu->arch.shared->msr & MSR_IS))
return 0;
gpa = get_tlb_raddr(tlbe);
......
arch/powerpc/kvm/emulate.c
View file @ 1765a1fe
......@@ -242,9 +242,11 @@ int kvmppc_emulate_instruction(struct kvm_run *run, struct kvm_vcpu *vcpu)
switch (sprn) {
case SPRN_SRR0:
kvmppc_set_gpr(vcpu, rt, vcpu->arch.srr0); break;
kvmppc_set_gpr(vcpu, rt, vcpu->arch.shared->srr0);
break;
case SPRN_SRR1:
kvmppc_set_gpr(vcpu, rt, vcpu->arch.srr1); break;
kvmppc_set_gpr(vcpu, rt, vcpu->arch.shared->srr1);
break;
case SPRN_PVR:
kvmppc_set_gpr(vcpu, rt, vcpu->arch.pvr); break;
case SPRN_PIR:
......@@ -261,13 +263,17 @@ int kvmppc_emulate_instruction(struct kvm_run *run, struct kvm_vcpu *vcpu)
kvmppc_set_gpr(vcpu, rt, get_tb()); break;
case SPRN_SPRG0:
kvmppc_set_gpr(vcpu, rt, vcpu->arch.sprg0); break;
kvmppc_set_gpr(vcpu, rt, vcpu->arch.shared->sprg0);
break;
case SPRN_SPRG1:
kvmppc_set_gpr(vcpu, rt, vcpu->arch.sprg1); break;
kvmppc_set_gpr(vcpu, rt, vcpu->arch.shared->sprg1);
break;
case SPRN_SPRG2:
kvmppc_set_gpr(vcpu, rt, vcpu->arch.sprg2); break;
kvmppc_set_gpr(vcpu, rt, vcpu->arch.shared->sprg2);
break;
case SPRN_SPRG3:
kvmppc_set_gpr(vcpu, rt, vcpu->arch.sprg3); break;
kvmppc_set_gpr(vcpu, rt, vcpu->arch.shared->sprg3);
break;
/* Note: SPRG4-7 are user-readable, so we don't get
* a trap. */
......@@ -320,9 +326,11 @@ int kvmppc_emulate_instruction(struct kvm_run *run, struct kvm_vcpu *vcpu)
rs = get_rs(inst);
switch (sprn) {
case SPRN_SRR0:
vcpu->arch.srr0 = kvmppc_get_gpr(vcpu, rs); break;
vcpu->arch.shared->srr0 = kvmppc_get_gpr(vcpu, rs);
break;
case SPRN_SRR1:
vcpu->arch.srr1 = kvmppc_get_gpr(vcpu, rs); break;
vcpu->arch.shared->srr1 = kvmppc_get_gpr(vcpu, rs);
break;
/* XXX We need to context-switch the timebase for
* watchdog and FIT. */
......@@ -337,13 +345,17 @@ int kvmppc_emulate_instruction(struct kvm_run *run, struct kvm_vcpu *vcpu)
break;
case SPRN_SPRG0:
vcpu->arch.sprg0 = kvmppc_get_gpr(vcpu, rs); break;
vcpu->arch.shared->sprg0 = kvmppc_get_gpr(vcpu, rs);
break;
case SPRN_SPRG1:
vcpu->arch.sprg1 = kvmppc_get_gpr(vcpu, rs); break;
vcpu->arch.shared->sprg1 = kvmppc_get_gpr(vcpu, rs);
break;
case SPRN_SPRG2:
vcpu->arch.sprg2 = kvmppc_get_gpr(vcpu, rs); break;
vcpu->arch.shared->sprg2 = kvmppc_get_gpr(vcpu, rs);
break;
case SPRN_SPRG3:
vcpu->arch.sprg3 = kvmppc_get_gpr(vcpu, rs); break;
vcpu->arch.shared->sprg3 = kvmppc_get_gpr(vcpu, rs);
break;
default:
emulated = kvmppc_core_emulate_mtspr(vcpu, sprn, rs);
......
arch/powerpc/kvm/powerpc.c
View file @ 1765a1fe
......@@ -38,9 +38,56 @@
int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
{
return !(v->arch.msr & MSR_WE) || !!(v->arch.pending_exceptions);
return !(v->arch.shared->msr & MSR_WE) ||
!!(v->arch.pending_exceptions);
}
int kvmppc_kvm_pv(struct kvm_vcpu *vcpu)
{
int nr = kvmppc_get_gpr(vcpu, 11);
int r;
unsigned long __maybe_unused param1 = kvmppc_get_gpr(vcpu, 3);
unsigned long __maybe_unused param2 = kvmppc_get_gpr(vcpu, 4);
unsigned long __maybe_unused param3 = kvmppc_get_gpr(vcpu, 5);
unsigned long __maybe_unused param4 = kvmppc_get_gpr(vcpu, 6);
unsigned long r2 = 0;
if (!(vcpu->arch.shared->msr & MSR_SF)) {
/* 32 bit mode */
param1 &= 0xffffffff;
param2 &= 0xffffffff;
param3 &= 0xffffffff;
param4 &= 0xffffffff;
}
switch (nr) {
case HC_VENDOR_KVM | KVM_HC_PPC_MAP_MAGIC_PAGE:
{
vcpu->arch.magic_page_pa = param1;
vcpu->arch.magic_page_ea = param2;
r2 = KVM_MAGIC_FEAT_SR;
r = HC_EV_SUCCESS;
break;
}
case HC_VENDOR_KVM | KVM_HC_FEATURES:
r = HC_EV_SUCCESS;
#if defined(CONFIG_PPC_BOOK3S) /* XXX Missing magic page on BookE */
r2 |= (1 << KVM_FEATURE_MAGIC_PAGE);
#endif
/* Second return value is in r4 */
break;
default:
r = HC_EV_UNIMPLEMENTED;
break;
}
kvmppc_set_gpr(vcpu, 4, r2);
return r;
}
int kvmppc_emulate_mmio(struct kvm_run *run, struct kvm_vcpu *vcpu)
{
......@@ -145,8 +192,10 @@ int kvm_dev_ioctl_check_extension(long ext)
case KVM_CAP_PPC_SEGSTATE:
case KVM_CAP_PPC_PAIRED_SINGLES:
case KVM_CAP_PPC_UNSET_IRQ:
case KVM_CAP_PPC_IRQ_LEVEL:
case KVM_CAP_ENABLE_CAP:
case KVM_CAP_PPC_OSI:
case KVM_CAP_PPC_GET_PVINFO:
r = 1;
break;
case KVM_CAP_COALESCED_MMIO:
......@@ -534,16 +583,53 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
return r;
}
static int kvm_vm_ioctl_get_pvinfo(struct kvm_ppc_pvinfo *pvinfo)
{
u32 inst_lis = 0x3c000000;
u32 inst_ori = 0x60000000;
u32 inst_nop = 0x60000000;
u32 inst_sc = 0x44000002;
u32 inst_imm_mask = 0xffff;
/*
* The hypercall to get into KVM from within guest context is as
* follows:
*
* lis r0, r0, KVM_SC_MAGIC_R0@h
* ori r0, KVM_SC_MAGIC_R0@l
* sc
* nop
*/
pvinfo->hcall[0] = inst_lis | ((KVM_SC_MAGIC_R0 >> 16) & inst_imm_mask);
pvinfo->hcall[1] = inst_ori | (KVM_SC_MAGIC_R0 & inst_imm_mask);
pvinfo->hcall[2] = inst_sc;
pvinfo->hcall[3] = inst_nop;
return 0;
}
long kvm_arch_vm_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg)
{
void __user *argp = (void __user *)arg;
long r;
switch (ioctl) {
case KVM_PPC_GET_PVINFO: {
struct kvm_ppc_pvinfo pvinfo;
r = kvm_vm_ioctl_get_pvinfo(&pvinfo);
if (copy_to_user(argp, &pvinfo, sizeof(pvinfo))) {
r = -EFAULT;
goto out;
}
break;
}
default:
r = -ENOTTY;
}
out:
return r;
}
......
arch/powerpc/kvm/trace.h
View file @ 1765a1fe
......@@ -98,6 +98,245 @@ TRACE_EVENT(kvm_gtlb_write,
__entry->word1, __entry->word2)
);
/*************************************************************************
* Book3S trace points *
*************************************************************************/
#ifdef CONFIG_PPC_BOOK3S
TRACE_EVENT(kvm_book3s_exit,
TP_PROTO(unsigned int exit_nr, struct kvm_vcpu *vcpu),
TP_ARGS(exit_nr, vcpu),
TP_STRUCT__entry(
__field( unsigned int, exit_nr )
__field( unsigned long, pc )
__field( unsigned long, msr )
__field( unsigned long, dar )
__field( unsigned long, srr1 )
),
TP_fast_assign(
__entry->exit_nr = exit_nr;
__entry->pc = kvmppc_get_pc(vcpu);
__entry->dar = kvmppc_get_fault_dar(vcpu);
__entry->msr = vcpu->arch.shared->msr;
__entry->srr1 = to_svcpu(vcpu)->shadow_srr1;
),
TP_printk("exit=0x%x | pc=0x%lx | msr=0x%lx | dar=0x%lx | srr1=0x%lx",
__entry->exit_nr, __entry->pc, __entry->msr, __entry->dar,
__entry->srr1)
);
TRACE_EVENT(kvm_book3s_reenter,
TP_PROTO(int r, struct kvm_vcpu *vcpu),
TP_ARGS(r, vcpu),
TP_STRUCT__entry(
__field( unsigned int, r )
__field( unsigned long, pc )
),
TP_fast_assign(
__entry->r = r;
__entry->pc = kvmppc_get_pc(vcpu);
),
TP_printk("reentry r=%d | pc=0x%lx", __entry->r, __entry->pc)
);
#ifdef CONFIG_PPC_BOOK3S_64
TRACE_EVENT(kvm_book3s_64_mmu_map,
TP_PROTO(int rflags, ulong hpteg, ulong va, pfn_t hpaddr,
struct kvmppc_pte *orig_pte),
TP_ARGS(rflags, hpteg, va, hpaddr, orig_pte),
TP_STRUCT__entry(
__field( unsigned char, flag_w )
__field( unsigned char, flag_x )
__field( unsigned long, eaddr )
__field( unsigned long, hpteg )
__field( unsigned long, va )
__field( unsigned long long, vpage )
__field( unsigned long, hpaddr )
),
TP_fast_assign(
__entry->flag_w = ((rflags & HPTE_R_PP) == 3) ? '-' : 'w';
__entry->flag_x = (rflags & HPTE_R_N) ? '-' : 'x';
__entry->eaddr = orig_pte->eaddr;
__entry->hpteg = hpteg;
__entry->va = va;
__entry->vpage = orig_pte->vpage;
__entry->hpaddr = hpaddr;
),
TP_printk("KVM: %c%c Map 0x%lx: [%lx] 0x%lx (0x%llx) -> %lx",
__entry->flag_w, __entry->flag_x, __entry->eaddr,
__entry->hpteg, __entry->va, __entry->vpage, __entry->hpaddr)
);
#endif /* CONFIG_PPC_BOOK3S_64 */
TRACE_EVENT(kvm_book3s_mmu_map,
TP_PROTO(struct hpte_cache *pte),
TP_ARGS(pte),
TP_STRUCT__entry(
__field( u64, host_va )
__field( u64, pfn )
__field( ulong, eaddr )
__field( u64, vpage )
__field( ulong, raddr )
__field( int, flags )
),
TP_fast_assign(
__entry->host_va = pte->host_va;
__entry->pfn = pte->pfn;
__entry->eaddr = pte->pte.eaddr;
__entry->vpage = pte->pte.vpage;
__entry->raddr = pte->pte.raddr;
__entry->flags = (pte->pte.may_read ? 0x4 : 0) |
(pte->pte.may_write ? 0x2 : 0) |
(pte->pte.may_execute ? 0x1 : 0);
),
TP_printk("Map: hva=%llx pfn=%llx ea=%lx vp=%llx ra=%lx [%x]",
__entry->host_va, __entry->pfn, __entry->eaddr,
__entry->vpage, __entry->raddr, __entry->flags)
);
TRACE_EVENT(kvm_book3s_mmu_invalidate,
TP_PROTO(struct hpte_cache *pte),
TP_ARGS(pte),
TP_STRUCT__entry(
__field( u64, host_va )
__field( u64, pfn )
__field( ulong, eaddr )
__field( u64, vpage )
__field( ulong, raddr )
__field( int, flags )
),
TP_fast_assign(
__entry->host_va = pte->host_va;
__entry->pfn = pte->pfn;
__entry->eaddr = pte->pte.eaddr;
__entry->vpage = pte->pte.vpage;
__entry->raddr = pte->pte.raddr;
__entry->flags = (pte->pte.may_read ? 0x4 : 0) |
(pte->pte.may_write ? 0x2 : 0) |
(pte->pte.may_execute ? 0x1 : 0);
),
TP_printk("Flush: hva=%llx pfn=%llx ea=%lx vp=%llx ra=%lx [%x]",
__entry->host_va, __entry->pfn, __entry->eaddr,
__entry->vpage, __entry->raddr, __entry->flags)
);
TRACE_EVENT(kvm_book3s_mmu_flush,
TP_PROTO(const char *type, struct kvm_vcpu *vcpu, unsigned long long p1,
unsigned long long p2),
TP_ARGS(type, vcpu, p1, p2),
TP_STRUCT__entry(
__field( int, count )
__field( unsigned long long, p1 )
__field( unsigned long long, p2 )
__field( const char *, type )
),
TP_fast_assign(
__entry->count = vcpu->arch.hpte_cache_count;
__entry->p1 = p1;
__entry->p2 = p2;
__entry->type = type;
),
TP_printk("Flush %d %sPTEs: %llx - %llx",
__entry->count, __entry->type, __entry->p1, __entry->p2)
);
TRACE_EVENT(kvm_book3s_slb_found,
TP_PROTO(unsigned long long gvsid, unsigned long long hvsid),
TP_ARGS(gvsid, hvsid),
TP_STRUCT__entry(
__field( unsigned long long, gvsid )
__field( unsigned long long, hvsid )
),
TP_fast_assign(
__entry->gvsid = gvsid;
__entry->hvsid = hvsid;
),
TP_printk("%llx -> %llx", __entry->gvsid, __entry->hvsid)
);
TRACE_EVENT(kvm_book3s_slb_fail,
TP_PROTO(u16 sid_map_mask, unsigned long long gvsid),
TP_ARGS(sid_map_mask, gvsid),
TP_STRUCT__entry(
__field( unsigned short, sid_map_mask )
__field( unsigned long long, gvsid )
),
TP_fast_assign(
__entry->sid_map_mask = sid_map_mask;
__entry->gvsid = gvsid;
),
TP_printk("%x/%x: %llx", __entry->sid_map_mask,
SID_MAP_MASK - __entry->sid_map_mask, __entry->gvsid)
);
TRACE_EVENT(kvm_book3s_slb_map,
TP_PROTO(u16 sid_map_mask, unsigned long long gvsid,
unsigned long long hvsid),
TP_ARGS(sid_map_mask, gvsid, hvsid),
TP_STRUCT__entry(
__field( unsigned short, sid_map_mask )
__field( unsigned long long, guest_vsid )
__field( unsigned long long, host_vsid )
),
TP_fast_assign(
__entry->sid_map_mask = sid_map_mask;
__entry->guest_vsid = gvsid;
__entry->host_vsid = hvsid;
),
TP_printk("%x: %llx -> %llx", __entry->sid_map_mask,
__entry->guest_vsid, __entry->host_vsid)
);
TRACE_EVENT(kvm_book3s_slbmte,
TP_PROTO(u64 slb_vsid, u64 slb_esid),
TP_ARGS(slb_vsid, slb_esid),
TP_STRUCT__entry(
__field( u64, slb_vsid )
__field( u64, slb_esid )
),
TP_fast_assign(
__entry->slb_vsid = slb_vsid;
__entry->slb_esid = slb_esid;
),
TP_printk("%llx, %llx", __entry->slb_vsid, __entry->slb_esid)
);
#endif /* CONFIG_PPC_BOOK3S */
#endif /* _TRACE_KVM_H */
/* This part must be outside protection */
......
arch/powerpc/platforms/Kconfig
View file @ 1765a1fe
......@@ -21,6 +21,16 @@ source "arch/powerpc/platforms/44x/Kconfig"
source "arch/powerpc/platforms/40x/Kconfig"
source "arch/powerpc/platforms/amigaone/Kconfig"
config KVM_GUEST
bool "KVM Guest support"
default y
---help---
This option enables various optimizations for running under the KVM
hypervisor. Overhead for the kernel when not running inside KVM should
be minimal.
In case of doubt, say Y
config PPC_NATIVE
bool
depends on 6xx || PPC64
......
arch/s390/include/asm/Kbuild
View file @ 1765a1fe
......@@ -5,6 +5,7 @@ header-y += chsc.h
header-y += cmb.h
header-y += dasd.h
header-y += debug.h
header-y += kvm_virtio.h
header-y += monwriter.h
header-y += qeth.h
header-y += schid.h
......
arch/s390/include/asm/kvm_virtio.h
View file @ 1765a1fe
......@@ -54,4 +54,11 @@ struct kvm_vqconfig {
* This is pagesize for historical reasons. */
#define KVM_S390_VIRTIO_RING_ALIGN 4096
/* These values are supposed to be in ext_params on an interrupt */
#define VIRTIO_PARAM_MASK 0xff
#define VIRTIO_PARAM_VRING_INTERRUPT 0x0
#define VIRTIO_PARAM_CONFIG_CHANGED 0x1
#define VIRTIO_PARAM_DEV_ADD 0x2
#endif
arch/x86/include/asm/kvm_emulate.h
View file @ 1765a1fe
......@@ -139,6 +139,7 @@ struct x86_emulate_ops {
void (*set_segment_selector)(u16 sel, int seg, struct kvm_vcpu *vcpu);
unsigned long (*get_cached_segment_base)(int seg, struct kvm_vcpu *vcpu);
void (*get_gdt)(struct desc_ptr *dt, struct kvm_vcpu *vcpu);
void (*get_idt)(struct desc_ptr *dt, struct kvm_vcpu *vcpu);
ulong (*get_cr)(int cr, struct kvm_vcpu *vcpu);
int (*set_cr)(int cr, ulong val, struct kvm_vcpu *vcpu);
int (*cpl)(struct kvm_vcpu *vcpu);
......@@ -156,7 +157,10 @@ struct operand {
unsigned long orig_val;
u64 orig_val64;
};
unsigned long *ptr;
union {
unsigned long *reg;
unsigned long mem;
} addr;
union {
unsigned long val;
u64 val64;
......@@ -190,6 +194,7 @@ struct decode_cache {
bool has_seg_override;
u8 seg_override;
unsigned int d;
int (*execute)(struct x86_emulate_ctxt *ctxt);
unsigned long regs[NR_VCPU_REGS];
unsigned long eip;
/* modrm */
......@@ -197,17 +202,16 @@ struct decode_cache {
u8 modrm_mod;
u8 modrm_reg;
u8 modrm_rm;
u8 use_modrm_ea;
u8 modrm_seg;
bool rip_relative;
unsigned long modrm_ea;
void *modrm_ptr;
unsigned long modrm_val;
struct fetch_cache fetch;
struct read_cache io_read;
struct read_cache mem_read;
};
struct x86_emulate_ctxt {
struct x86_emulate_ops *ops;
/* Register state before/after emulation. */
struct kvm_vcpu *vcpu;
......@@ -220,12 +224,11 @@ struct x86_emulate_ctxt {
/* interruptibility state, as a result of execution of STI or MOV SS */
int interruptibility;
bool restart; /* restart string instruction after writeback */
bool perm_ok; /* do not check permissions if true */
int exception; /* exception that happens during emulation or -1 */
u32 error_code; /* error code for exception */
bool error_code_valid;
unsigned long cr2; /* faulted address in case of #PF */
/* decode cache */
struct decode_cache decode;
......@@ -249,13 +252,14 @@ struct x86_emulate_ctxt {
#define X86EMUL_MODE_HOST X86EMUL_MODE_PROT64
#endif
int x86_decode_insn(struct x86_emulate_ctxt *ctxt,
struct x86_emulate_ops *ops);
int x86_emulate_insn(struct x86_emulate_ctxt *ctxt,
struct x86_emulate_ops *ops);
int x86_decode_insn(struct x86_emulate_ctxt *ctxt);
#define EMULATION_FAILED -1
#define EMULATION_OK 0
#define EMULATION_RESTART 1
int x86_emulate_insn(struct x86_emulate_ctxt *ctxt);
int emulator_task_switch(struct x86_emulate_ctxt *ctxt,
struct x86_emulate_ops *ops,
u16 tss_selector, int reason,
bool has_error_code, u32 error_code);
int emulate_int_real(struct x86_emulate_ctxt *ctxt,
struct x86_emulate_ops *ops, int irq);
#endif /* _ASM_X86_KVM_X86_EMULATE_H */
arch/x86/include/asm/kvm_host.h
View file @ 1765a1fe
This diff is collapsed.
arch/x86/include/asm/kvm_para.h
View file @ 1765a1fe
......@@ -158,6 +158,12 @@ static inline unsigned int kvm_arch_para_features(void)
return cpuid_eax(KVM_CPUID_FEATURES);
}
#ifdef CONFIG_KVM_GUEST
void __init kvm_guest_init(void);
#else
#define kvm_guest_init() do { } while (0)
#endif
#endif /* __KERNEL__ */
#endif /* _ASM_X86_KVM_PARA_H */
arch/x86/include/asm/msr-index.h
View file @ 1765a1fe
......@@ -198,6 +198,7 @@
#define MSR_IA32_TSC 0x00000010
#define MSR_IA32_PLATFORM_ID 0x00000017
#define MSR_IA32_EBL_CR_POWERON 0x0000002a
#define MSR_EBC_FREQUENCY_ID 0x0000002c
#define MSR_IA32_FEATURE_CONTROL 0x0000003a
#define FEATURE_CONTROL_LOCKED (1<<0)
......
arch/x86/include/asm/pvclock.h
View file @ 1765a1fe
......@@ -12,4 +12,42 @@ void pvclock_read_wallclock(struct pvclock_wall_clock *wall,
struct pvclock_vcpu_time_info *vcpu,
struct timespec *ts);
/*
* Scale a 64-bit delta by scaling and multiplying by a 32-bit fraction,
* yielding a 64-bit result.
*/
static inline u64 pvclock_scale_delta(u64 delta, u32 mul_frac, int shift)
{
u64 product;
#ifdef __i386__
u32 tmp1, tmp2;
#endif
if (shift < 0)
delta >>= -shift;
else
delta <<= shift;
#ifdef __i386__
__asm__ (
"mul %5 ; "
"mov %4,%%eax ; "
"mov %%edx,%4 ; "
"mul %5 ; "
"xor %5,%5 ; "
"add %4,%%eax ; "
"adc %5,%%edx ; "
: "=A" (product), "=r" (tmp1), "=r" (tmp2)
: "a" ((u32)delta), "1" ((u32)(delta >> 32)), "2" (mul_frac) );
#elif defined(__x86_64__)
__asm__ (
"mul %%rdx ; shrd $32,%%rdx,%%rax"
: "=a" (product) : "0" (delta), "d" ((u64)mul_frac) );
#else
#error implement me!
#endif
return product;
}
#endif /* _ASM_X86_PVCLOCK_H */
arch/x86/kernel/kvmclock.c
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kernel/pvclock.c
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kvm/Kconfig
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kvm/emulate.c
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kvm/i8254.c
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kvm/i8259.c
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kvm/irq.c
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kvm/irq.h
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kvm/kvm_cache_regs.h
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kvm/lapic.c
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kvm/mmu.c
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kvm/mmu.h
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kvm/mmu_audit.c 0 → 100644
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kvm/mmutrace.h
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kvm/paging_tmpl.h
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kvm/svm.c
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kvm/timer.c
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kvm/vmx.c
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kvm/x86.c
View file @ 1765a1fe
This diff is collapsed.
arch/x86/kvm/x86.h
View file @ 1765a1fe
This diff is collapsed.
drivers/s390/kvm/kvm_virtio.c
View file @ 1765a1fe
This diff is collapsed.
include/linux/kvm.h
View file @ 1765a1fe
This diff is collapsed.
include/linux/kvm_host.h
View file @ 1765a1fe
This diff is collapsed.
include/linux/kvm_para.h
View file @ 1765a1fe
This diff is collapsed.
mm/util.c
View file @ 1765a1fe
This diff is collapsed.
virt/kvm/irq_comm.c
View file @ 1765a1fe
This diff is collapsed.
virt/kvm/kvm_main.c
View file @ 1765a1fe
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7