From d73eb57b80b98ae147e4e6a7d9877c2ba175f972 Mon Sep 17 00:00:00 2001 From: Wanpeng Li Date: Thu, 18 Jul 2019 19:39:06 +0800 Subject: KVM: Boost vCPUs that are delivering interrupts Inspired by commit 9cac38dd5d (KVM/s390: Set preempted flag during vcpu wakeup and interrupt delivery), we want to also boost not just lock holders but also vCPUs that are delivering interrupts. Most smp_call_function_many calls are synchronous, so the IPI target vCPUs are also good yield candidates. This patch introduces vcpu->ready to boost vCPUs during wakeup and interrupt delivery time; unlike s390 we do not reuse vcpu->preempted so that voluntarily preempted vCPUs are taken into account by kvm_vcpu_on_spin, but vmx_vcpu_pi_put is not affected (VT-d PI handles voluntary preemption separately, in pi_pre_block). Testing on 80 HT 2 socket Xeon Skylake server, with 80 vCPUs VM 80GB RAM: ebizzy -M vanilla boosting improved 1VM 21443 23520 9% 2VM 2800 8000 180% 3VM 1800 3100 72% Testing on my Haswell desktop 8 HT, with 8 vCPUs VM 8GB RAM, two VMs, one running ebizzy -M, the other running 'stress --cpu 2': w/ boosting + w/o pv sched yield(vanilla) vanilla boosting improved 1570 4000 155% w/ boosting + w/ pv sched yield(vanilla) vanilla boosting improved 1844 5157 179% w/o boosting, perf top in VM: 72.33% [kernel] [k] smp_call_function_many 4.22% [kernel] [k] call_function_i 3.71% [kernel] [k] async_page_fault w/ boosting, perf top in VM: 38.43% [kernel] [k] smp_call_function_many 6.31% [kernel] [k] async_page_fault 6.13% libc-2.23.so [.] __memcpy_avx_unaligned 4.88% [kernel] [k] call_function_interrupt Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Christian Borntraeger Cc: Paul Mackerras Cc: Marc Zyngier Signed-off-by: Wanpeng Li Signed-off-by: Paolo Bonzini --- arch/s390/kvm/interrupt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'arch/s390') diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c index 9dde4d7d8704..26f8bf4a22a7 100644 --- a/arch/s390/kvm/interrupt.c +++ b/arch/s390/kvm/interrupt.c @@ -1240,7 +1240,7 @@ void kvm_s390_vcpu_wakeup(struct kvm_vcpu *vcpu) * The vcpu gave up the cpu voluntarily, mark it as a good * yield-candidate. */ - vcpu->preempted = true; + vcpu->ready = true; swake_up_one(&vcpu->wq); vcpu->stat.halt_wakeup++; } -- cgit v1.2.3-55-g7522 From d984740944308a310f9d33df774e2304fc1e6959 Mon Sep 17 00:00:00 2001 From: Wanpeng Li Date: Thu, 18 Jul 2019 19:39:07 +0800 Subject: KVM: s390: Use kvm_vcpu_wake_up in kvm_s390_vcpu_wakeup Use kvm_vcpu_wake_up() in kvm_s390_vcpu_wakeup(). Suggested-by: Paolo Bonzini Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Christian Borntraeger Signed-off-by: Wanpeng Li Signed-off-by: Paolo Bonzini --- arch/s390/kvm/interrupt.c | 23 +++-------------------- 1 file changed, 3 insertions(+), 20 deletions(-) (limited to 'arch/s390') diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c index 26f8bf4a22a7..b5fd6e85657c 100644 --- a/arch/s390/kvm/interrupt.c +++ b/arch/s390/kvm/interrupt.c @@ -1224,28 +1224,11 @@ no_timer: void kvm_s390_vcpu_wakeup(struct kvm_vcpu *vcpu) { - /* - * We cannot move this into the if, as the CPU might be already - * in kvm_vcpu_block without having the waitqueue set (polling) - */ vcpu->valid_wakeup = true; + kvm_vcpu_wake_up(vcpu); + /* - * This is mostly to document, that the read in swait_active could - * be moved before other stores, leading to subtle races. - * All current users do not store or use an atomic like update - */ - smp_mb__after_atomic(); - if (swait_active(&vcpu->wq)) { - /* - * The vcpu gave up the cpu voluntarily, mark it as a good - * yield-candidate. - */ - vcpu->ready = true; - swake_up_one(&vcpu->wq); - vcpu->stat.halt_wakeup++; - } - /* - * The VCPU might not be sleeping but is executing the VSIE. Let's + * The VCPU might not be sleeping but rather executing VSIE. Let's * kick it, so it leaves the SIE to process the request. */ kvm_s390_vsie_kick(vcpu); -- cgit v1.2.3-55-g7522