Use the analysis tool to determine the VSImax, which is the maximum capacity of the tested system expressed in the amount of Login VSI sessions. The Login VSI analyzer can be installed on one of the launcher machines but can also be installed at a standalone machine as long as the “VSI Share” is available.
This video contains captions (subtitles)
1. Calculating VSIMax
Typically the desktop workload is scripted in a 12-14 minute loop when a simulated Login VSI user is logged on. After the loop is finished it will restart automatically. Within each loop the response times of seven specific operations is measured in a regular interval: six times in within each loop. The response times if these seven operations are used to establish VSImax. The seven operations from which the response times are measured are:
Copy new doc from the document pool in the home drive
This operation will refresh a new document to be used for measuring the response time. This activity is mostly a file-system operation.
Starting Microsoft Word with a document
This operation will measure the responsiveness of the Operating System and the file system. Microsoft Word is started and loaded into memory, also the new document is automatically loaded into Microsoft Word. When the disk I/O is extensive or even saturated, this will impact the file open dialogue considerably.
Starting the “File Open” dialogue
This operation is handled for small part by Word and a large part by the operating system. The file open dialogue uses generic subsystems and interface components of the OS. The OS provides the contents of this dialogue.
Starting “Notepad”
This operation is handled by the OS (loading and initiating notepad.exe) and by the Notepad.exe itself through execution. This operation seems instant from an end-user’s point of view.
Starting the “Print” dialogue
This operation is handled for a large part by the OS subsystems, as the print dialogue is provided by the OS. This dialogue loads the print-subsystem and the drivers of the selected printer. As a result, this dialogue is also dependent on disk performance.
Starting the “Search and Replace” dialogue \
This operation is handled within the application completely; the presentation of the dialogue is almost instant. Serious bottlenecks on application level will impact the speed of this dialogue.
Compress the document into a zip file with 7-zip command line
This operation is handled by the command line version of 7-zip. The compression will very briefly spike CPU and disk I/O.
These measured operations with Login VSI do hit considerably different subsystems such as CPU (user and kernel), Memory, Disk, the OS in general, the application itself, print, GDI, etc. These operations are specifically short by nature. When such operations are consistently long: the system is saturated because of excessive queuing on any kind of resource. As a result, the average response times will then escalate. This effect is clearly visible to end-users. When such operations consistently consume multiple seconds the user will regard the system as slow and unresponsive. With Login VSI 3.0 it is now possible to choose between ‘VSImax Classic’ and 'VSImax Dynamic’.
1.1. VSIMax Classic
VSImax Classic is based on the previous versions of Login VSI, and is achieved when the average Login VSI response time is higher than a fixed threshold of 4000ms. This method proves to be reliable when no anti-virus or application virtualization is used. If you are using VSI 3.0 or later VSIMax dynamic is the recommended analysis method. To calculate the response times the seven activities listed in the previous section are totaled. To balance these measurements are weighted before they are summed. Without weighting individual response times before they are totaled, one specific measurement (out of seven) could dominate the results. Within ‘VSImax Classic’ two measurements are weighted before they are added to the total VSImax response time:
‘Starting Microsoft Word with a document’ is divided by two (50%)
‘Starting the “Search and Replace” dialogue’ is multiplied by five (500%)
A sample of the VSImax Classic response time calculation is displayed below:
Then the average VSImax response time is calculated based on the amount of active Login VSI users logged on to the system. When the average VSImax response times are consistently higher than the default threshold of 4000ms, VSImax is achieved. In practice however, tests have shown a substantial increase of application response time when antivirus and/or application virtualization is used. The baseline response time is typically around 1400 - 1800 ms without application virtualization or antivirus. However, when anti-virus or application virtualization is used, the baseline response time varies between 2500 – 3500 ms. When the baseline response time is already so high the VSImax threshold of 4000ms is too easily reached. ‘VSImax Classic’ will report a maximum long before system resources like CPU, mem or disk are actually saturated. It was therefore decided further optimize VSImax calculation. Now in Login VSI 3.0 ‘VSImax Dynamic’ is introduced to be able to support wildly varying baseline response times when anti-virus and/or application virtualization is used.
1.2. VSIMax Dynamic
Similar to ‘VSImax Classic’, VSImax Dynamic is calculated when the response times are consistently above a certain threshold. However, this threshold is now dynamically calculated on the baseline response time of the test.
Five individual measurements are weighted to better support this approach:
Copy new doc from the document pool in the home drive: 100%
Microsoft Word with a document: 33.3%
Starting the “File Open” dialogue: 100%
Starting “Notepad”: 300%
Starting the “Print” dialogue: 200%
Starting the “Search and Replace” dialogue: 400%
Compress the document into a zip file with 7-zip command line 200%
A sample of the VSImax Dynamic response time calculation is displayed below:
Then the average VSImax response time is calculated based on the amount of active Login VSI users logged on to the system. For this the average VSImax response times need to consistently higher than a dynamically calculated threshold. To determine this dynamic threshold, first the average baseline response time is calculated. This is done by averaging the baseline response time of the first 15 Login VSI users on the system. The formula for the dynamic threshold is: Avg. Baseline Response Time x 125% + 3000. As a result, when the baseline response time is 1800, the VSImax threshold will now be 1800 x 125% + 3000 = 5250ms. Especially when application virtualization is used, the baseline response time can wildly vary per vendor and streaming strategy. Therefore it is recommend to use VSImax Dynamic when comparisons are made with application virtualization or anti-virus agents. The result VSImax Dynamic scores are aligned again with saturation on a CPU, Memory or Disk level, also when the baseline response time are relatively high.
2. Determing VSIMax
The Login VSI analyser will automatically identify the “VSImax”. In the example below the VSImax is 98. The analyser will automatically determine “stuck sessions” and correct the final VSImax score.
Vertical axis: Response Time in milliseconds
Horizontal axis: Total Active Sessions
Red line: Maximum Response (worst response time of an individual measurement within a single session)
Orange line: Average Response Time within for each level of active sessions
Blue line: the VSImax average.
Green line: Minimum Response (best response time of an individual measurement within a single session)
3. Automated Analysis
The Login VSI analyser is capable of analysing results from the commandline, this is extermely usefull when executing a lot of tests in an automated test environment. To generate the testresults via the commandline on a default Login VSI setup use the following parameters:
C:\Program Files\Login Consultants\VSI\Analyzer\VSI Analyzer Console.exe "Name of the test" "Output.csv" [-Classic]
First parameter: Name of the test (Used when starting the test)
Second parameter: Output file name to append the results to
Third parameter: Force classic analysis (Not recommended!)
4. Compare Wizard
The Login VSI Compare Wizard enables to merge data from multiple tests in one chart to help visualize differences between tests.
This video contains captions (subtitles)
