Calculating Maximum Virtual Desktop Capacity – VSImax Explained
In virtualized desktop environments, knowing how many users any given configuration can support is critical to maintaining uptime, delivering on SLAs and providing a great desktop experience for end users. Load testing and stress testing virtualized desktop environments is a key use case for Login VSI. After running a Login VSI load test, the Analyzer provides information on the maximum capacity of your virtualized desktop environment with a number--VSImax. But you may ask how this number is actually calculated. This is an excellent question because understanding what goes into a VSImax can help you to better interpret the test results in the Login VSI Analyzer.
In this blog post I will explain the philosophy behind VSImax and I will use our preshipped "Example – Test – Office – 2013" test results to show how we calculate the baseline, threshold and the system saturation point known as VSImax.
This blog is divided into several sections:
The philosophy behind Login VSI is distinctive from conventional benchmarks. In general, most system benchmarks are steady state benchmarks. These benchmarks execute one or multiple processes, and the measured execution time is the outcome of the test. Simply put, the faster the execution time or the bigger the throughput, the faster the system is according to the benchmark.
Login VSI has a different approach and was designed to perform benchmarks for XenApp / RDS or VDI workloads through system saturation by simulating realistic user workloads. The goal of Login VSI is to find what the maximum desktop/user session capacity is of a given system. Login VSI loads the system with simulated user workloads using well known desktop applications like Microsoft Office, Internet Explorer and Adobe Acrobat Reader. The most popular workload is the Knowledge worker workload.
By gradually increasing the number of simulated users, the system will eventually be saturated. Once the system is saturated, the response time of the applications will increase significantly. This latency in session response times is a clear indication whether the system is (close to being) overloaded. As a result, by nearly overloading a system it is possible to find out what its true maximum user capacity is.
The problem is the behavior of common applications. For instance, the Office applications used may be different between organizations. Applications get upgraded over time and even behave differently when they are started a second time within the session. This is why Login VSI uses the specially designed VSImax measurements also called Timers to gauge the response time of the (desktop) session instead of individual application response times.
When the system is coming closer to its saturation point, response times will rise. When reviewing the average response time it will be clear the VSI response times escalate at saturation point. After a test is performed, the VSI response times can be analyzed to calculate the maximum active session/desktop capacity. Within Login VSI this is calculated the VSImax.
This VSImax is defined as the Virtual Session Index (VSI) for Virtual Desktop Infrastructure (VDI) and Terminal Services (RDS) workloads. This index simplifies comparisons and makes it possible to understand the true impact of configuration changes on hypervisor host or guest level.
To fully understand the calculation behind VSImax, we will dig into these topics:
During a test, each session measures the session responsiveness by running timers. These timers consist of five metrics that measure the session responsiveness in different ways:
- Notepad Start Load (NLSD)
- Notepad File Open (NFO)
- Zip High Compression (ZHC)
- Zip Low Compression (ZLC)
Each metric will be weighted and the total sum of these results represents the total session response time for that given session during that time. If there are more sessions active in the environment it will take longer for these metrics to finish.
The results of these timers can be found in the Raw Data table. These are the timer metrics and explained how they measure session responsiveness.
|Timer Metrics||Scale Weights|
|Notepad Start Load (NSLD)||0.2|
|Loading and initiating VSINotepad.exe (a .Net Application) and opening a file. This operation is also handled by the OS and by the VSINotepad.exe itself through execution. This operation seems almost instant from an end-user’s point of view.|
|Notepad File Open (NFO)||0.75|
|Loading and initiating VSINotepad.exe and opening the openfile dialog. This operation is handled by the OS and by the VSINotepad.exe itself through execution. This operation seems almost instant from an end-user’s point of view.|
|Zip High Compression (ZHC)||0.125|
|This action copy's a random file and compresses it (with 7zip) with high compression enabled. The compression will very briefly spike CPU and disk IO.|
|Zip Low Compression (ZLC)||0.2|
|This action copy's a random file and compresses it (with 7zip) with low compression enabled. The compression will very briefly disk IO and creates some load on the CPU as well.|
|Calculates a large array of random data and spikes the CPU for a short period of time.|
After measuring the average response time with the timers, the Analyzer collects this data from the Raw Data Table and calculates the VSIindex Average. This is the blue line in the VSImax graph below and shows the system weighted response time (Y-Axis) during a given number of active sessions (X-Axis).
Below you will see a small portion of the Raw Data table. Each test done with Login VSI and opened up in the Login VSI Analyzer will have one.
Raw Data Table
A. To calculate the average response time for the amount of active sessions. The analyzer takes the first timer report entry for every session count and every entry that was reported 90 seconds before this one.
Example: To calculate the VSIindex Average for sessions count 22 the analyzer takes the first entry with sessions count 22 and every report 90 seconds before that one.
B. Continuing with this selection the Analyzer multiplies each timer metric by the scale weights. For each timer entry the total sum of these weighted metrics are roundup and entered in the total_response column.
Raw Data table weighted
C. Then the total_response column for the same selection is sorted by lowest to highest. If there are more than 7 values in this range, the analyzer will trim these bottom and top by at least 1 or 5% (whichever is higher).
Raw Data table sorted
Example: If you have 12 values (5% of 12 = 0.6) the Analyzer will trim one from the top and bottom. Which will leave you with 10 entries. Then the average of these remaining values is calculated and noted as the VSI Index Calculation for session count 22.
The above index calculation is repeated for all the first entries for every session count. The results of all the VSI Index Calculations can be found in the VSImax v4 Data Table in the Analyzer (see the image on the right side). The VSIIndex_Calculations will be visualized as the blue line in the VSImax v4 graph.
VSImax v4 data table
After calculating the VSIindex, the Analyzer will calculate the saturation point (VSImax). To locate this saturation point, the Login VSI Analyzer will first analyze the baseline and the threshold.
To set a saturation point (the threshold) the Analyzer first measures a baseline. This number represents the average session response time with minimum load on it. The baseline is calculated by taking the average of the 13 lowest VSI Index Calculations in the entire test.
Raw Data table Baseline
Example: This table shows you the 13 lowest VSI Index Calculations of the entire test. The average of these results is 1336 and is set as the baseline for this particular test.
Secondly, the Threshold can be calculated by adding 1000ms to the baseline.
Lastly, the VSImax index is calculated. The session count number of the first VSIIndex_Calculation that exceeds the threshold is subtracted by 1 and set as VSImax.
Example: According to the previous steps the baseline was 1336 and the threshold was set at 2336. In this example on the right you can see that the first VSIindex Calculation that exceeds our threshold is 2339. And thus we subtract 1 from the corresponding session count and we have our VSImax of 114 sessions.
VSImax with a maximum capacity of 114 sessions
VSImax is a very useful metric to see how many users are supported by your virtualized desktop environment, but how is this number calculated by Login VSI? Timers consisting of five metrics collect data during a test and this data is visible in the Raw Data table in the Analyzer. The Analyzer uses this data to calculate the VSI index average, this number shows the system weighted response time during a given amount of active sessions. After this, the Analyzer can set the saturation point by first calculating the baseline by taking the average of the 13 lowest VSI Index Calculations in the entire test and setting a threshold by adding 1000ms to this baseline. The saturation point (VSImax) is the session count before the threshold was reached.
I hope that this blog has helped you to understand the philosophy and calculation of VSImax. If you have any questions, please do not hesitate to reach out to us at firstname.lastname@example.org.