Top 12 Virtualized Desktop Performance Testing Mistakes
You've stepped into the great world of virtualized desktops. You know performance is key to end-user experience. Either you've already learned or you are about to learn that performance testing in this world is different. Schadenfreude. For all the other people who have made performance testing mistakes, you can scoff at their errors as you reap the benefits.
In this article I want to run you through the top 12 issues that we see happening when people start to performance test their virtual desktop environment with Login VSI.
- Neglect to test at all
- Starting with a full-scale test
- Testing only with own applications and workloads
- Not knowing what you’re testing
- Assuming tests scale linearly
- Improperly configuring the load test
- Not diving deeper into VSImax
- Disregarding stuck sessions
- Only capturing Login VSI data
- Not rebooting
- Using balanced power settings
- Not testing in production
It sounds strange, but actually not executing any performance testing is the biggest mistake we can think of and poses the most risk (to system downtime and to project success). In last year’s Project VRC survey, 44% of survey respondents said that they never perform any load tests. I have seen many environments where even the simplest performance test prior to having real users could log on could have prevented catastrophe. A few examples that I have personally witnessed:
- A VDI environment that went down just above 4,000 users because the environment consumed too much power
- Multiple VDI environments that went down because of provisioning servers with too little capacity
- Storage systems that went down under load because of errors in firmware
- VDI environments wherein only 1 out of 20 CPUs were utilized.
- Updating Microsoft Office 2010 to 2013 and losing 20% of users per server.
Why start with a 20 or 50 user test if you can run a 1,000 user one? Sure a 1,000 user test sounds a lot more interesting, but let’s first check if you have set up everything correctly by doing some smaller scale practice runs before going full board. Once you’ve mastered a 50 or smaller user test, scale up to test 1 full server. Then, why stop there? Test your full environment. At every step along the way you will likely be making improvements to your environment that help you reach your end-goal.
It is certainly valuable to test your own applications and workloads, and Login VSI allows you to do so. However, just testing your own applications limits you. I recommend to get started with the out-of-the-box applications just to save you a lot of time and effort setting up minimal workloads. Login VSI workloads are being seen as the standard in the industry allowing you to not only test very quickly, but also to compare your results to the performance claims of vendors in their published whitepapers. This blog with 5 best practices on workload customization could be very helpful.
You can customize workloads in Login VSI but for your first test we recommend using the standard workloads.
So you thought you would build and test a VDI or Server Based Computing environment in one week with limited experience working in and with these complex environments? Sure, it can be done. Follow some online how-to’s and you can get up and running in no time. However, when executing a load test, deeper knowledge of these environments will help you to isolate the problem a lot quicker.
I've tested one server, so I know how this will scale. Nope. Starting your tests with just one server is great, but after that you will need to scale things up to find the bigger problems in your infrastructure. Typically these are found in shared components such as storage, networking, or databases. This blog post outlines why VDI does not scale linearly and why you should test full-scale to locate the bottlenecks in your environment.
VDI uses shared components and unfortunately, does not scale linearly.
You wouldn’t be the first to configure Login VSI in such a way that it will try to launch a 1,000 user session in 1 second. While Login VSI is perfectly capable of doing this, it’s unlikely that you have the infrastructure that can live up to that dream. Therefore, configure a number of users and launch window that’s realistic. I’m not saying that you should not stress the system but keep the load within the realm of the realistic.
I have reached my VSImax, and now I have nothing more to check. Hold your horses, cowboy. VSImax is a great way of indicating the maximum number of users your environment can support before performance starts to degrade. Fine, but wouldn’t you also like to know what caused the environment to be maxed out? Also, do not forget to check your baseline response. Baseline is a great indicator of system performance when there is little to no stress. A high baseline typically means your users are suffering throughout the day, even before hitting the VSImax.
If you did not reach your VSImax, it doesn't mean your test failed. Quite the opposite. Not reaching VSImax means that the number of users you tested with will fit on the environment you tested. This is actually good news. You could test with more users than you liked but if you are fine with the outcome, why push it?
Check the baseline in the VSImax chart for additional information about the performance of your environment.
Remember you are doing a stress test. Something is bound to go wrong at some point. If just a few sessions (on a 1000 user test) get stuck, I wouldn’t worry too much. But as soon as you see lots of stuck sessions, it indicates that your test is becoming invalid. Are you overloading your systems? Is your test configured properly? Find the reason for your stuck sessions and see if there is something you can do to prevent them.
When analyzing complex performance issues, the more information you can get your hands on the better. Therefore I always recommend running performance metrics capture tools at every level of your infrastructure during tests. Whether it’s on your storage system, hypervisor (e.g. vSphere or XenServer) or plain old Perfmon, collecting these results and combining them within Login VSI’s Analyzer gives you a clear overview of where problems exist and at how many users they start to show.
The VSImax chart with external performance data
When doing a performance test, you are stressing your hardware and software infrastructure. Make sure to reboot all of the machines related to the test after every run. After that, also make sure you test as scientifically as possible. For example, after a reboot, always wait for the same amount of time for the machines to ‘cool down’ before you start a test as this can influence your results.
Some things should be balanced: your life, your diet and the tires on your car. Your VDI servers should not be set to balanced power mode. Time and time again we see poor performance of VDI environments being caused by this one simple setting. Make sure to configure the power settings in the BIOS of your servers correctly and give your users the performance they deserve.
My environment is in production now therefore I can stop testing. Sorry, this is not such a good idea. Every change you introduce into your production environment should be thoroughly tested. Some changes have a major impact on the scalability of systems and on the performance that end-users experience. Running Login VSI tests on a regular basis before bringing the change to production can prevent a lot of errors from reaching users. This blog about the VDI lifecycle can help to guarantee the best performance in production.
The VDI lifecycle can help you to guarantee a good performance in your test and production environment.
As you can see, there are lots of details to look out for when doing performance tests in your virtualized desktop environment. Hopefully you can now get ahead of the game and benefit from others' performance testing mistakes. Our mission at Login VSI is to make performance testing as easy as possible. If you want to try it out for yourself, go to the Hands-on Labs or download your trial today. And if you run into any issues, our support team is always here to help you.