login vsi company logo login vsi company logo 250x40
header-05.jpg
  • Home
  • Blog
  • Login VSI
  • Testing Meltdown and Spectre patches: Citrix XenServer 7.2 versus 7.5

Testing Meltdown and Spectre patches: Citrix XenServer 7.2 versus 7.5

Meltdown and Spectre

A few weeks ago we published a blogpost around Meltdown and Spectre on XenServer 7.2. In the previous blogpost I mentioned that the results were surprisingly good and other then expected. This feeling kept bothering me and urged me to verify if the behavior and results were correct. After some research I found out that the patches were applied and that both used tools (Powershell and Inspectre.exe) to check this verified this.  I decided to install newer versions of XenServer to see if the results and behavior was consistent to what we saw with XenServer 7.2. I chose to focus on XenServer 7.5 as this is the latest version that XenServer currently supplies. I also updated it to the latest patch version (XS75E002). 

My goal was to verify if I could get results, with XenServer 7.5, that were in-line with the expected impact reported in the market OR that the first tests on XenServer 7.2 were correct. In the table underneath, you can find the different runs that I have tested and are compared to each other.

Meltdown Spectre

Applied Patches:

ScenarioAdded Patch
Clean Windows 2012R2 feb. 2018 updates / Windows Server 2016 (Update feb 2018)
Microsoft Patch (MS) KB4056898 for 2012R2, KB4056890 for 2016
Hypervisor Patches (HV) XS72E017 and XS75E001 / XS75E002
Microcode Patch, BIOS update (MCU) Dell R730 Bios ver. 2.7.1

The results

As I hoped I did find results that were expected from the market: there is a definitive impact with Meltdown and Spectre patches on XenServer. It surprises me that I did not see similar behavior in both XenServer versions and it raises questions.

In the graphs below you can find the visual representations of the test results. In the graphs you can clearly see that the Meltdown and Spectre patches have a big impact on the system and see that there is a big difference between 7.5 and 7.2. I have compared all tests with the “clean” environment as well. As this has no patches applied you can correctly see the impact of all patches. All results are compared with the clean run done on XenServer 7.2.

As a side note, in the initial test with the XS72E017 patch I only tested with 2012r2. After I noticed that there was no major difference with the XS72E015 patch I chose not to run the same test with 2016. And continue with installing XenServer 7.5.

VSIMax XenServer
VSImax Results, higher is better

VSIBase XenServer
VSIbase results, lower is better

Percentile Impact 2012R2
2012R2 % impact closest to 0 is best

Percentile Impact 2016
2016 % Impact closest to 0 is best

Conclusions

The results of the tests are bad and good. On one side I am happy I found the Meltdown and Spectre impact on the XenServer installation. An impact that is in line with market expectations. On the other side I did expect to find the performance impact on both XenServer 7.2 and 7.5 however this is not the case. This worries me a bit and raises a few new questions:

  1. Why does the full-stack patch on 7.2 not show the performance impact?
    1. Is this a clever trick or are the patches not applied in some way?
      1. Does that mean XenServer 7.2 users are not protected?
    2. How can we verify this, when all used checks confirmed the patch application.
  2. Which XenServer version changes the impact of the Meltdown and Spectre patches, 7.3, 7.4, or 7.5?

About the Lab used for these tests

Automate, automate, automate. In our VDILIKEAPRO test lab we have minimized human interaction (and thus error) by making sure all processes are executed fully automatic. For example the base images are created using the Microsoft Deployment Toolkit where as a next step PowerShell magic (thanks Henk & Sonny!) takes over and automatically clones the VM’s 6 times on a single host, prepares them for use, and runs 10 automated test cycles. Results are then averaged ignoring the first run.

All the connections to the target machines are done using the RDP protocol. And the Login VSI default workload for this type of testing, the Knowledge Worker workload, was used for all tests.

VDILIKEAPRO Lab

Physical Virtual Infra

 

Our industry-standard software is built to help you avoid problems, lower costs and improve performance. Request a Quote or get your (free!) Trial below, and benefit from our award-winning services:

Request Quote Request Trial

 

 

About the author

Tom Willemsen is a Support Engineer at Login VSI and helps customers and gives advice about testing with Login VSI. He loves to travel, read about history, and see different cultures. In his free time, Tom likes to game and watch movies.


Tags: VSImax, Spectre, Meltdown, VSIbase

Cookie Settings