[Bug 1764956] Re: Guests using IBRS incur a large performance penalty

I've managed to verify this on kernel 4.4.0-143-generic, the problem
isn't reproducible anymore.

Worth noticing that when using the kernel 4.4.0-143 in both host and
guest, we cannot change MSR bit anymore from within the guest, so the
performance drop is not observed.

** Tags removed: verification-needed-xenial
** Tags added: verification-done-xenial

--
You received this bug notification because you are subscribed to linux
in Ubuntu.
Matching subscriptions: Bgg, Bmail, Nb
https://bugs.launchpad.net/bugs/1764956

Title:
Guests using IBRS incur a large performance penalty

Status in linux package in Ubuntu:
In Progress
Status in linux source package in Trusty:
In Progress
Status in linux source package in Xenial:
Fix Committed

Bug description:
[Impact]
the IBRS would be mistakenly enabled in the host when the switching
from an IBRS-enabled VM and that causes the performance overhead in
the host. The other condition could also mistakenly disables the IBRS
in VM when context-switching from the host. And this could be
considered a CVE host.

[Fix]
The patch fixes the logic inside the x86_virt_spec_ctrl that it checks
the ibrs_enabled and _or_ the hostval with the SPEC_CTRL_IBRS as the
x86_spec_ctrl_base by default is zero. Because the upstream
implementation is not equal to the Xenial's implementation. Upstream
doesn't use the IBRS as the formal fix. So, by default, it's zero.

On the other hand, after the VM exit, the SPEC_CTRL register also
needs to be saved manually by reading the SPEC_CTRL MSR as the MSR
intercept is disabled by default in the hardware_setup(v4.4) and
vmx_init(v3.13). The access to SPEC_CTRL MSR in VM is direct and
doesn't trigger a trap. So, the vmx_set_msr() function isn't called.

The v3.13 kernel hasn't been tested. However, the patch can be viewed
at:
http://kernel.ubuntu.com/git/gavinguo/ubuntu-trusty-amd64.git/log/?h=sf00191076-sru

The v4.4 patch:
http://kernel.ubuntu.com/git/gavinguo/ubuntu-xenial.git/log/?h=sf00191076-spectre-v2-regres-backport-juerg

[Test]

The patch has been tested on the 4.4.0-140.166 and works fine.

The reproducing environment:
Guest kernel version: 4.4.0-138.164
Host kernel version: 4.4.0-140.166

(host IBRS, guest IBRS)

- 1). (0, 1).
The case can be reproduced by the following instructions:
guest$ echo 1 | sudo tee /proc/sys/kernel/ibrs_enabled
1

<Several minutes later...>

host$ cat /proc/sys/kernel/ibrs_enabled
0
host$ for i in {0..55}; do sudo rdmsr 0x48 -p $i; done
11111111111111000000000000000000010010100000000000000000

Some of the IBRS bit inside the SPEC_CTRL MSR are mistakenly
enabled.

host$ taskset -c 5 stress-ng -c 1 --cpu-ops 2500
stress-ng: info: [11264] defaulting to a 86400 second run per stressor
stress-ng: info: [11264] dispatching hogs: 1 cpu
stress-ng: info: [11264] cache allocate: default cache size: 35840K
stress-ng: info: [11264] successful run completed in 33.48s

The host kernel didn't notice the IBRS bit is enabled. So, the situation
is the same as "echo 2 > /proc/sys/kernel/ibrs_enabled" in the host.
And running the stress-ng is a pure userspace CPU capability
calculation. So, the performance downgrades to about 1/3. Without the
IBRS enabled, it needs about 10s.

- 2). (1, 1) disables IBRS in host -> (0, 1) actually it becomes (0, 0).
The guest IBRS has been mistakenly disabled.

guest$ echo 2 | sudo tee /proc/sys/kernel/ibrs_enabled
guest$ for i in {0..55}; do sudo rdmsr 0x48 -p $i; done
11111111111111111111111111111111111111111111111111111111

host$ echo 2 | sudo tee /proc/sys/kernel/ibrs_enabled
host$ for i in {0..55}; do sudo rdmsr 0x48 -p $i; done
11111111111111111111111111111111111111111111111111111111
host$ echo 0 | sudo tee /proc/sys/kernel/ibrs_enabled
host$ for i in {0..55}; do sudo rdmsr 0x48 -p $i; done
00000000000000000000000000000000000000000000000000000000

guest$ for i in {0..55}; do sudo rdmsr 0x48 -p $i; done
00000000000000000000000000000000000000000000000000000000

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1764956/+subscriptions