How-To: Update, Protect Against RIDL, Fallout MDS Vulnerability
RIDL and Fallout MDS vulnerabilities, impact on VDI performance & actions to take.
This article provides you with the most common questions we’re receiving and updates on performance tests currently being executed to patch the flaws Intel refers to as: “Microarchitectural Data Sampling (M.D.S.)” a.k.a.: Rogue In-Flight Data Load (RIDL), Fallout, ZombieLoad and Store-to-Leak Forwarding.
It has been quiet for a while - on the surface at least - but security researchers have not stopped investigating Intel CPU’s for new flaws. Yesterday new security issues have been made public (shout out to the fellow Dutchman who discovered this flaw) that Intel refers to as: “Microarchitectural Data Sampling (M.D.S.)”. Immediately WhatsApp, Slack and email started to buzz notifying me something was up.
In this article - that will be updated as we go - I’ll be providing you with the most common questions we’re receiving and updates on performance tests currently being executed. Additionally, this article contains the steps executed to patch our lab, to use as a reference, if you’re in the process of updating yourself, as little documentation is available currently. As multiple research teams have discovered different flaws you might have seen the alternative names: Rogue In-Flight Data Load (RIDL), Fallout, ZombieLoad and Store-to-Leak Forwarding.
Please note that I’m not a security expert, my tips are focused on performance, not safety!
There are many articles outlining the theoretical and practical implications of testing here’s a few good examples:
- Citrix blog post: Microarchitectural data sampling security issues and mitigations
- VMware ESXi 6.7, Patch Release ESXi670-201905001
- Powershell Gallery: SpeculationControl/1.0.13
- ADV190013 | Microsoft Guidance to mitigate Microarchitectural Data Sampling vulnerabilities
If you are running a VDI environment today e.g. VMware Horizon, or Citrix Virtual Apps and Desktops, what are immediate steps to take, and how does it affect performance and scalability?
As most of Login VSI’s customers are running the combination VMware as hypervisor, with Citrix on top for desktop delivery, that’s our focus for now.
VMware has been kind enough to create the image below and I couldn’t have done a better job myself as it perfectly shows the need for capacity testing/planning before going live. These mitigations can have a huge impact on how many users you can run in your datacenter.
As I’m lucky enough to have a lab-environment at hand (see details) I’ve set off to update VMware vCenter first. The update process is simple and only took a few minutes (for reference, this environment consists of approximately 10 hosts and about 3000 VM’s, of which 400 where powered on with users logged on, these users where not affected and could continue working. To update your own environment go to your vCenter appliance management portal https://vCenter.FQDN.com: 5480.
From there search for updates, pre-stage and Install.
After the vCenter update completes, and an automatic reboot of the services, the next step is to patch my ESXi servers. Naturally I’m not going to do this by hand, so I’ll leverage the update mechanism available. In your vSphere client browse to the “Update manager”.
By choosing new in the baselines tab, I can create a new patch baseline. By selecting the 14th of May as a date in the wizard, I can quickly select the updates needed. I’ve only selected the 6.7 updates as my lab is running the latest versions. If you are running older versions make sure to select the correct ones.
After creating the baseline we’ve got to attach it to a system, to do so find the host(s) you’d like to test on or mitigate and attach the baseline in the updates tab. This will cause the system to be validated against the baseline prompting it is “not compliant”.
After running the pre-checks, click remediate to start the patching process on the hypervisor.
As seen in the VMware flowchart this doesn’t automatically enable the mitigations. Allowing us to test the impact of different settings. Please note that mitigations are only effective when all layers (Microcode, Hypervisor and operating system) have been patched. This is the same for all hypervisors.
When using VMware ESXi you might note there are multiple versions of mitigations as in ESXi 6.7u2 SCAv2 was released and offers performance improvements over SCAv1 while protecting from VM to VM and VM to Hypervisor information leakage. The side-channel aware scheduler has been enhanced with a new policy to allow hyper-threads to be used concurrently if both threads are running vCPU contexts from the same VM. In this way, L1TF side channels are constrained to not expose information across VM/VM or VM/hypervisor boundaries. Source: https://blogs.vmware.com/performance/author/todd_muirhead
To download the specific hotfix by Microsoft you must search the windows update catalog (or download all updates available).
- Windows 10 1803:
So what’s next?
Simply run through the process of change impact testing. These will be the tests that I’ll be executing in the upcoming time, sharing my results and updating this article. Please note that if you are updating your own environment, mileage may vary as this is a lab environment. If you’re hosting your environment in the cloud I can highly recommend taking a look at that with Login PI to safeguard performance.
Start using Login VSI today
Our industry-standard software is built to help you avoid problems, lower costs and improve performance. Request a Quote or get your (free!) Trial below, and benefit from our award-winning services.
About the company Login VSI
The company Login VSI provides end-user performance insights for virtualized desktop and server based computing environments. Enterprise IT departments use flagship product Login VSI (for load testing) and new addition Login PI (for continuity testing) in all phases of their virtual desktop deployment—from planning to deployment to change management—to build and safeguard a good performance, a high availability, and (as a result) a good and consistent end-user experience. For more information about Login VSI or for a free test license contact us.