login vsi company logo login vsi company logo 250x40
header-05.jpg

Top 12 Virtualized Desktop Performance Testing Mistakes

Top 12 Virtualized Desktop Performance Testing Mistakes

You've stepped into the great world of virtualized desktops. You know performance is key to end-user experience. Either you've already learned or you are about to learn that performance testing in this world is different. Schadenfreude. For all the other people who have made performance testing mistakes, you can scoff at their errors as you reap the benefits.

In this article I want to run you through the top 12 issues that we see happening when people start to performance test their virtual desktop environment with Login VSI.

  1. Neglect to test at all
  2. Starting with a full-scale test
  3. Testing only with own applications and workloads
  4. Not knowing what you’re testing
  5. Assuming tests scale linearly
  6. Improperly configuring the load test
  7. Not diving deeper into VSImax
  8. Disregarding stuck sessions
  9. Only capturing Login VSI data
  10. Not rebooting
  11. Using balanced power settings
  12. Not testing in production

1. Neglect to test at all

It sounds strange, but actually not executing any performance testing is the biggest mistake we can think of and poses the most risk (to system downtime and to project success). In last year’s Project VRC survey, 44% of survey respondents said that they never perform any load tests. I have seen many environments where even the simplest performance test prior to having real users could log on could have prevented catastrophe. A few examples that I have personally witnessed:

  • A VDI environment that went down just above 4,000 users because the environment consumed too much power
  • Multiple VDI environments that went down because of provisioning servers with too little capacity
  • Storage systems that went down under load because of errors in firmware
  • VDI environments wherein only 1 out of 20 CPUs were utilized.
  • Updating Microsoft Office 2010 to 2013 and losing 20% of users per server.

2. Starting with a full-scale test

Why start with a 20 or 50 user test if you can run a 1,000 user one? Sure a 1,000 user test sounds a lot more interesting, but let’s first check if you have set up everything correctly by doing some smaller scale practice runs before going full board. Once you’ve mastered a 50 or smaller user test, scale up to test 1 full server. Then, why stop there? Test your full environment. At every step along the way you will likely be making improvements to your environment that help you reach your end-goal.

3. Testing only with own applications and workloads

It is certainly valuable to test your own applications and workloads, and Login VSI allows you to do so. However, just testing your own applications limits you. I recommend to get started with the out-of-the-box applications just to save you a lot of time and effort setting up minimal workloads. Login VSI workloads are being seen as the standard in the industry allowing you to not only test very quickly, but also to compare your results to the performance claims of vendors in their published whitepapers. This blog with 5 best practices on workload customization could be very helpful. 

top 12 virtualized desktop performance testing mistakes workload customization

You can customize workloads in Login VSI but for your first test we recommend using the standard workloads.

4. Not knowing what you're testing

So you thought you would build and test a VDI or Server Based Computing environment in one week with limited experience working in and with these complex environments? Sure, it can be done. Follow some online how-to’s and you can get up and running in no time. However, when executing a load test, deeper knowledge of these environments will help you to isolate the problem a lot quicker.

5. Assuming tests scale linearly

I've tested one server, so I know how this will scale. Nope. Starting your tests with just one server is great, but after that you will need to scale things up to find the bigger problems in your infrastructure. Typically these are found in shared components such as storage, networking, or databases. This blog post outlines why VDI does not scale linearly and why you should test full-scale to locate the bottlenecks in your environment.

top 12 virtualized desktop performance testing mistakes vdi does not scale linearly

VDI uses shared components and unfortunately, does not scale linearly.

6. Improperly configuring the load test

You wouldn’t be the first to configure Login VSI in such a way that it will try to launch a 1,000 user session in 1 second. While Login VSI is perfectly capable of doing this, it’s unlikely that you have the infrastructure that can live up to that dream. Therefore, configure a number of users and launch window that’s realistic. I’m not saying that you should not stress the system but keep the load within the realm of the realistic.

7. Not diving deeper into VSImax

I have reached my VSImax, and now I have nothing more to check. Hold your horses, cowboy. VSImax is a great way of indicating the maximum number of users your environment can support before performance starts to degrade. Fine, but wouldn’t you also like to know what caused the environment to be maxed out? Also, do not forget to check your baseline response. Baseline is a great indicator of system performance when there is little to no stress. A high baseline typically means your users are suffering throughout the day, even before hitting the VSImax.

If you did not reach your VSImax, it doesn't mean your test failed. Quite the opposite. Not reaching VSImax means that the number of users you tested with will fit on the environment you tested. This is actually good news. You could test with more users than you liked but if you are fine with the outcome, why push it?

top 12 virtualized desktop performance testing mistakes vsimax

Check the baseline in the VSImax chart for additional information about the performance of your environment.

8. Disregarding stuck sessions

Remember you are doing a stress test. Something is bound to go wrong at some point. If just a few sessions (on a 1000 user test) get stuck, I wouldn’t worry too much. But as soon as you see lots of stuck sessions, it indicates that your test is becoming invalid. Are you overloading your systems? Is your test configured properly? Find the reason for your stuck sessions and see if there is something you can do to prevent them.

9. Only capturing Login VSI data

When analyzing complex performance issues, the more information you can get your hands on the better. Therefore I always recommend running performance metrics capture tools at every level of your infrastructure during tests. Whether it’s on your storage system, hypervisor (e.g. vSphere or XenServer) or plain old Perfmon, collecting these results and combining them within Login VSI’s Analyzer gives you a clear overview of where problems exist and at how many users they start to show.

top 12 virtualized desktop performance testing mistakes vsimax external performance data

The VSImax chart with external performance data

10. Not rebooting

When doing a performance test, you are stressing your hardware and software infrastructure. Make sure to reboot all of the machines related to the test after every run. After that, also make sure you test as scientifically as possible. For example, after a reboot, always wait for the same amount of time for the machines to ‘cool down’ before you start a test as this can influence your results.

11. Using balanced power settings

Some things should be balanced: your life, your diet and the tires on your car. Your VDI servers should not be set to balanced power mode. Time and time again we see poor performance of VDI environments being caused by this one simple setting. Make sure to configure the power settings in the BIOS of your servers correctly and give your users the performance they deserve.

12. Not testing in production

My environment is in production now therefore I can stop testing. Sorry, this is not such a good idea. Every change you introduce into your production environment should be thoroughly tested. Some changes have a major impact on the scalability of systems and on the performance that end-users experience. Running Login VSI tests on a regular basis before bringing the change to production can prevent a lot of errors from reaching users. This blog about the VDI lifecycle can help to guarantee the best performance in production.

top 12 virtualized desktop performance testing mistakes the vdi lifecycle

The VDI lifecycle can help you to guarantee a good performance in your test and production environment.

Conclusion

As you can see, there are lots of details to look out for when doing performance tests in your virtualized desktop environment. Hopefully you can now get ahead of the game and benefit from others' performance testing mistakes. Our mission at Login VSI is to make performance testing as easy as possible. If you want to try it out for yourself, go to the Hands-on Labs or download your trial today. And if you run into any issues, our support team is always here to help you.

About the author

Mark Plettenberg (@markplettenberg) is a product manager of Login VSI and has played a critical role in the development and growth of Login VSI. Ask Mark about motorcycle mechanics and breaking/repairing anything and everything that has a power plug.


Tags: How-to, Login VSI, #VDILIKEAPRO, Load Testing, Workloads, Best Practices, VSImax, Support

Start Delivering the Best End User Experience Today

Request a Demo

Login VSI, Inc.

3945 Freedom Circle
Suite 670
Santa Clara, CA 95054

Phone: +1 408 899 7418

300 Tradecenter
Suite 3460
Woburn, MA 01801

Phone: +1 408 899 7418

Login VSI B.V.

De Entree 85
1101 BH Amsterdam
The Netherlands

Phone: +31 20 705 1200