login vsi company logo login vsi company logo 250x40

 

Get the best possible VDI performance, density and availability with Login VSI
Our Solutions

Testing L1TF patches: Virtual Desktops on VMware ESX

Last week Intel announced 3 new severe vulnerabilities in their processors allowing unauthorized access to the data in the L1-cache. They have been named L1 Terminal Fault or in short L1TF. By now I assume most IT-admins are aware of this, but the performance impact remains a mystery. That’s why we took it upon us to get you this information as soon as possible.

Please note that these are initial findings. As we are doing more research and get more results we will constantly publish new updates.

CVEName
CVE-2018-3615 L1 Terminal Fault-SGX
CVE-2018-3620 L1 Terminal Fault-OS/ SMM
CVE-2018-3646 L1 Terminal Fault-VMM

In this article I’m going to focus on the third variant (3646) as I expect it to have the biggest impact on the scalability and performance of Virtual Desktop environments such as VMware Horizon View and Citrix XenApp / XenDesktop. For those of you on AMD CPU’s there is good news, as they do not seem to be affected.

How can this leak be exploited? Simply said a malicious virtual machine (VM) running on a certain CPU core can access privileged information of another VM that is on the same CPU core at the same time and read its L1 Data Cache. This is possible because Intel processors share physically addressed L1 Data Cache across both logical processors of a Hyperthreading enabled core. When patching is not an option a quick way to mitigate this could be to disable hyperthreading although that might have a significant impact on performance or cause capacity issues and is therefore discouraged by VMware:“Disabling Intel Hyperthreading in firmware/BIOS (or by using VMkernel.Boot.Hyperthreading) after applying vSphere updates and patches is not recommended and precludes potential vSphere scheduler enhancements and mitigations that will allow the use of both logical processors. As such, disablement of hyperthreading to mitigate the Concurrent-context attack vector will introduce unnecessary operational overhead as hyperthreading may need to be re-enabled in the future.”

I have conducted my tests on VMware ESXi 6.5.0 Update 2 (Build 9298722) and my preliminary research is focused on the VMkernel.Boot.HyperthreadingMitigation setting that restricts the simultaneous use of logical processors from the same hyperthreaded core as necessary to mitigate a security vulnerability. This is the most reliable way to prevent exploits as all virtual machines are considered untrusted siblings. This level of security is needed e.g. in cloud desktop environments and high secure environments as financial institutions or hospitals. Meanwhile my college Tom is performing these tests on Citrix XenServer, but more on that later.

L1TF
IMAGE SOURCE: VMware

According to VMware it is safe to patch vCenter and/or the ESXi hosts as the mitigation is disabled by default. This is a great way to get ready for the next step: researching capacity issues. Naturally I am using the industry standard load testing solution Login VSI to simulate users on my environment. To start I have installed the patch but have left VMkernel.Boot.HyperthreadingMitigation to its default setting: False. A friendly message notifies me of this setting after the update is complete.

L1TF 2

As my previous tests where on Server 2016 I decided to start measuring the impact on that platform as it would save some setup time. While testing bare RDSH machines it is expected that the relative impact will be similar on Citrix XenApp and Horizon shared session hosts but make sure to validate this in your own environments.

I have deployed 6 Windows server 2016 machines with 4vCPU’s and 55GB of memory resulting in an environment that could run 196 users before VSImax was hit. This is the maximum number of users that can work on an environment before performance becomes a bottleneck. Interestingly enabling HTMitigation did not impact performance too much, at first this had us wondering but discussions with performance experts quickly led us to conclude that the number of VM’s and vCPU’s simply allowed the hypervisor to work out a scenario where core’s where not shared.

 VSImaxVSIbaseVSIavg
Windows Server 2016 (Default) 196 700 1677
Windows Server 2016 with HTM enabled 193 694 1678

L1TF 3

So, we changed the configuration, now running with 8 Server 2016 machines each with 32GB of memory and 6vCPU’s. This slightly lowered VSImax to 186 users.

 VSImaxVSIbaseVSIavg
Windows Server 2016 (Default) 196 700 1677
Windows Server 2016 with HTM enabled 186 694 1532

L1TF 4

L1TF 5

All right, knowing this it was time to step up the game and switch to Windows 10. I started out with a fresh copy of build 17134.1 (1803) with no further Windows updates and gave it a spin. In our lab we deployed a 180 VM’s al equipped with 2GB of memory and 2 vCPU’s and kicked of a test. As you can see the VSImax drops approximately 20%.

 VSImaxVSIbaseVSIavg
Windows 10 140 1023 1977
Windows 10 with HTMitigation enabled 110 1004 1963

L1TF 6

As you can see the performance/density? hit is significant, there are however nuances: different operating systems, newer (or older) CPU’s and of course the applications and infrastructure in your own environment will be of influence on the exact impact. In addition: It also seems that the impact of the L1TF patch depends heavily on your configuration. When using RDS machines in an efficient way (when hyperthreading is fully utilized) the patch has minimal impact. However, we do see a bigger impact when you do utilize the Hyperthreading tech i.e. VDI.

Please note that these are the first results - many thanks to Jasper Geelen for the help and assistance - and we still have many questions remaining so updates are to be expected. If you like to get more info feel free to reach out, or if you’d like to test your own environment: Download your trial of Login VSI today.

 


 

Start using Login VSI today

Our industry-standard software is built to help you avoid problems, lower costs and improve performance. Request a Quote or get your free Trial below, and benefit from our award-winning services.

Free Price Request  Your Free Trial

 


About the company Login VSI

The company Login VSI provides end-user performance insights for virtualized desktop and server-based computing environments. Enterprise IT departments use flagship product Login VSI (for load testing) and Login PI (for continuity testing) in all phases of their virtual desktop deployment—from planning to deployment to change management—to build and safeguard a good performance, a high availability, and (as a result) a good and consistent end-user experience. For more information about Login VSI or for a free test license contact us.

 

About the author

Mark Plettenberg (@markplettenberg) is a product manager of Login VSI and has played a critical role in the development and growth of Login VSI. Ask Mark about motorcycle mechanics and breaking/repairing anything and everything that has a power plug.


Tags: News, Login VSI

Popular Blogs

Login VSI Blog - Ongoing Effects CPU Flaws

The Ongoing Effects of Intel CPU Flaws

The Ongoing Effects of Intel CPU Flaws Over the last year, we’ve seen many Intel CPU hardware flaws come to light and when news first broke about Meltdown & Spectre there was a lot of panic: "should we patch?", "What’s the performance impact?", "Can we still rely on this hardware?" Continue Reading
Login VSI Blog - How-To Update, Protect Against RIDL, Fallout MDS Vulnerability

How-To: Update, Protect Against RIDL, Fallout MDS Vulnerability

RIDL & Fallout MDS vulnerabilities, impact on VDI performance & actions to take. FAQs we’re receiving & updates on performance tests executed to patch flaws Intel calls “Microarchitectural Data Sampling (M.D.S.)” aka: Rogue In-Flight Data Load (RIDL), Fallout, ZombieLoad & Store-to-Leak Forwarding. Continue Reading
Login VSI Blog - Teaser Image - A Practical Guide to VDI Change Management - Part 1

A Practical Guide to VDI Change Management

Part 1: IT Change Management in general The first in an 8-part series, this practical guide to VDI Change Management will guide you through the transformation of the IT department from a back-end function into a core competency for every modern organization. Continue Reading
Login VSI - Blog - Login PI Blog Teaser Image - Windows Virtual Desktop: How To Monitor User Experience With Login PI

Windows Virtual Desktop – How to Monitor User Experience?

Microsoft has just announced the public preview of their new Windows Virtual Desktop (WVD) offering at Microsoft Ignite on Tour in Amsterdam today. For those of you who’ve not followed the rumors or the private beta, here’s the outline... Continue Reading
A Practical Guide To VDI Change Management, Part 3

A Practical Guide to VDI Change Management

Part 3: Change Accelerates with Windows 10 The third in an 8-part series, this practical guide to VDI Change Management gives you the low-down on the what, why and how of Windows 10 updates and changes Continue Reading
Login VSI Blog - Teaser Image - A Practical Guide to VDI Change Management - Part 2

A Practical Guide to VDI Change Management

Part 2: Why VDI is very sensitive to change The second in an 8-part series, this practical guide to VDI Change Management highlights the importance of effective Change Management. With all the complexities of VDI environments, any failure can severely impact your business. Continue Reading
Cookie Settings