Further Analyzing The Intel CPU "x86 PTI Issue" On More Systems

2018 has been off to a busy start with all the testing around the Linux x86 PTI (Page Table Isolation) patches for this "Intel CPU bug" that potentially dates back to the Pentium days but has yet to be fully disclosed. Here is the latest.

Yesterday I posted the first benchmarks of the performance impact of these x86 PTI security changes that landed in the Linux 4.15 kernel just days ago. As outlined in that article, most of the slowdowns attributed to the page table isolation come down to slower I/O but not universally as it largely depends upon the I/O workload as well as the speed of the actual storage device. In most desktop-ish workloads, the impact of enabling x86 PTI is much less like with not seeing much of a change for gaming.

Since that article yesterday I've been running kernel tests using the latest custom-built kernel code on about two dozen different systems (testing made easy thanks to the Phoronix Test Suite and Phoromatic) from the latest Kabylake and Coffeelake hardware and Xeon Scalable servers to old pre-SandyBridge boxes. The latest results largely jive with my initial findings and the most interesting bits now shared in this article.

More benchmarks have been running, including on some old non-PCID hardware and Ryzen. pic.twitter.com/wjc16Eilxn

— Michael Larabel (@michaellarabel) January 3, 2018

While with the Linux development code currently, AMD CPUs are marked as insecure and those PTI applied, as covered earlier today, being staged via the tip tree is the much talked about AMD patch. But if that patch will land in time for this month's Linux 4.15 kernel release or be held off until the Linux 4.16 kernel cycle remains to be seen. Regardless, that AMD patch will end up landing some point soon so AMD CPU owners won't be negatively impacted as their hardware appears immune to this latest security issue.

The latest on the situation:

- There's been many headlines today on other websites proclaiming this Intel CPU bug yields "30% slowdowns" as if it's something universal or even close to being universal. I have yet to find any very common workload where end-users would be dramatically impacted: the I/O slowdowns are when incurring heavy I/O synthetic benchmarking on fast NVMe solid-state drives. Some of the synthetic I/O examples:

- When testing on AMD Ryzen but with PTI active, indeed, there is a similar performance hit to Intel. But if using a mainline kernel until that patch ends up being there, just reiterating you can boot your kernel with the "nopti" kernel parameter. Intel users can also opt for the nopti switch if they want to retain maximum performance, but it's a potential security risk. The Ryzen impact:

- Desktop systems were easy to illustrate the performance difference with the synthetic I/O tests and database workloads like PostgreSQL. When running tests on programs not making frequent use of system calls or other kernel interactions, the performance impact tended to be nominal.

- Reiterating from yesterday's article, systems having PCID (Process Context ID) should lessen the impact of PTI being enabled. (Those interested can check for the presence of "pcid" in their "/proc/cpuinfo" output.) PCID has been present on Intel hardware since the Westmere days, so basically any Sandy Bridge era system or newer should be in better shape. I did manage to pull out an old Lenovo ThinkPad W510 with an Intel Core i7 720QM Clarksfield that is from 2009 and lacks PCID but is affected by this cpu_insecure issue.

- On that old Clarksfield-era ThinkPad I wasn't going to be surprised if the performance was disastrous, but it wound up being better than I had anticipated given all the ongoing drama... In general purpose workloads there was no reportable performance difference in our frequent benchmark test cases. Under I/O, the PTI-using kernel did yield some slower results but not by the margins seen on the newer systems with faster storage. The laptop consumer-grade HDD in this laptop appeared to be the main bottleneck and kernel inefficiencies weren't causing as dramatic slowdowns.

- To some surprise, when carrying out network benchmarks with netperf/iperf3, in at least those contexts PTI didn't have a noticeable impact on the network throughput performance.

- To see if your system is impacted (but it basically comes down to being Intel x86 CPUs or temporarily for AMD CPUs) can check for "cpu_insecure" on the bug line in the /proc/cpuinfo output if running the Linux 4.15-rc6 or later. There is also the CONFIG_PAGE_TABLE_ISOLATION Kconfig switch for fully enabling the functionality, and that's looking like it will default to yes by time of Linux 4.16.

- There has been KAISER/PTI patches for AArch64 / 64-bit ARM but yet to be merged. It's unclear if there is any hardware risk at this point or if it's solely a preventive measure.

- On Xeon Scalable class hardware the I/O performance slowdowns when running the tests bare metal on the host tended to generally be just a few percent slower on affected benchmarks, less than with the "desktop" hardware.

- What's most serious about this situation is the CPU bug and how long it's been present now in seemingly all modern Intel CPUs and plagues all major operating systems. But in terms of the performance impact of addressing this issue in software, while any performance slowdown is unfortunate, at least for the workloads tested thus far they've basically been manageable. I would equate it similarly to the cost of enjoying full-disk encryption... While I am a self-admitted performance junkie, on any production system with sensitive data I always make use of full-disk encryption as it's simply worthwhile for the added layer of physical protection at the small cost of performance. Page table isolation makes sense in increasing kernel security: it's not that this is some outright shoddy code hack to workaround a bug but something that does make sense.

For those curious about the raw numbers, the latest batch will be wrapped up shortly.