Adventures in Creating a Common Storage Workload
With Login VSI’s role as the industry-standard in performance testing within centralized desktop virtualization, it’s no surprise that storage-related vendors would come to us to develop a workload specific to storage. Testing that requires less infrastructure is a specific requirement for storage vendors because it allows for larger-scale tests. The goal for the storage workload is simple: Increase the number of IOPs per user (while being gentle on CPU and memory).
A simple job?
On the surface this appeared to be a relatively straightforward assignment for Login VSI. And a useful one: Agreement on storage workloads would help make more sense of the rapidly changing storage market. There have been many new developments in storage capacity adding to confusion. What’s more, competing vendors each offer a slightly different interpretation of storage capacity or needs.
Not so fast
It turns out that creating a storage workload is not easy or straightforward at all (even for us—ha ha). We found that our general assumptions about workloads didn’t translate to the storage world. However, we were able to overcome many obstacles and developed a working Storage Workload beta that we been testing with several storage vendors.
Those familiar with Login VSI know about workloads. In creating the Storage Workload, we used the generic medium workload (Knowledge Worker) and replaced the low IPOs/high CPU applications and activities (like watching a video or typing a document) with more IOPS-intensive activities. Additionally, we:
- Dramatically increased the speed of the virtual user/worker (to an unrealistic level)
- Built in a mechanism to continuously empty the cache (so that apps cannot start from memory)
- Allowed the Storage Workload to run in the so-called Direct Desktop mode, without the need for launchers. (This takes away the influence of the remoting protocol in the test which is less relevant for an IOPS-intensive storage test).
With these changes some things stayed the same: We still do everything with real data, real applications and virtual users.
We also learned an interesting fact: There is a maximum number of IOPS that can be reached with real data and real applications. Without “going synthetic” we are roughly reaching the 25 to 30 IOPS point, which is about 3 times the amount in our normal medium workload. These results will vary depending on performance tuning in your environment.
Is this enough? Do we need to add a volume knob to mix in synthetic workload and pump up the volume?
Most vendors claim that this is enough and that Login VSI should remain “the purist of performance testing.” There are enough synthetic workload providers. Real data with real applications are what truly make Login VSI different.
As more storage vendors experience the Storage Workload beta, we will collect and interpret feedback. Please be aware that the Storage Workload, once out of beta, will not be included in Login VSI Professional License offering, but will instead be an add-on with separate pricing.