Storage Latency: Is fast storage really faster?

November 20th, 2014 by

I have an idea… Faster storage isn’t always faster. Before you disagree let me explain. There are a lot of different factors that go into the “speed” of your storage. The first thing you have to do is define speed. In this case I’ll be looking at it from the end users perspective. Any one of a variety of factors could have an effect on this “speed” and storage latency. It could be the network type FC vs iSCSI, or the disk speed SATA vs SSD, or the controller, or even the utilization of VMs on the pool that drive the “speed” of the storage. You can’t look at one of these factors in isolation to make decisions about what is your fast storage and what is your slow storage.

storage latency fast and slow lanes

Let me give you an example. I was recently working with a large government client in New York. We were looking through some of the decisions that VMTurbo was making. Over 100 of these decisions were around storage latency, and they said to me I’d never want to move that machine from my SAS LUN to the SATA LUN.

Being the guy that has to question these types of comments I asked “why?” although the answer is always the same. The SAS drives are the “fast” storage and that is a critical machine so I would never put it on “slow” storage. But, back to our previous point… This logic is inherently flawed. Although there are more IOPS on the SAS drives the storage access latency was higher due to the majority of workloads running on the SAS LUNs. Take a look below.

stroage latency SAS vs SATA

Technically yes the SAS drive spins faster than the SATA drive. Technically yes the SAS drive can handle more IOPS than the SATA drive. But, let’s think about this for a second. Aren’t there other factors that affect the performance of storage other than just the speed of the disks? Any good storage admin could probably name off ten. The one big one that stands out to me instantly is the demand for storage from the virtual machines.

We can’t just look at the equation as critical VMs go on fast storage, non-critical VMs go on slow storage. How many VMs do you have on the fast storage? How many VMs do you have on the slow storage? Are there too many VMs all trying to get resources from the fast storage, and almost none trying to get resources from the slow storage?

At a high level the question we should really be asking is, “What is the supply of storage available from the SAN, and what is the demand for storage coming from the virtual machine?” By answering this question as opposed to arbitrarily assigning VMs to some class of storage we can make sure that ALL of the VMs are able to access the storage resources they need. The best answer isn’t always to put everything on fast storage, the best answer is always to make sure the VMs have access to the resources they demand in order to reduce the amount of storage latency experienced by the end user.

By allowing some critical VMs to move over to SATA drives from SAS drives this customer was actually able to increase the performance of ALL of the VMs within the environment (see the state below).

stroage latency SAS vs SATA after vmt

This is because they stopped thinking about where are the most IOPS available and started using VMTurbo to build out a picture of the supply of storage resources on the SAN and the demand for storage resources from the VMs, and took the actions to make sure that supply meets demand.

While the total percentage utilization was increased on the SATA drives we were still able to support the other workloads without drastically increasing latency. We were also able to significantly reduce the amount of latency on the SAS drives due to factors outside of IOPS utilization.

On a side note, in this case it was due to the storage controller utilization that we were able to see such drastic results, but it could have been any number of factors.

At the end of the day this is a simple concept that’s impossible to manage in real time in a large or even medium sized environment. We looked at one factor, disk speed, for one environment. But, as we discussed at the beginning there are multiple factors that affect storage.

Even if you could identify all of the factors associated with storage performance, and I’m sure some of you could. What would you do with the data? By the time you finished analyzing it would the data still be relevant? This is why VMTurbo defines these relationships in software and analyzes them in real time.

VMTurbo’s unique demand driven approach starts with what resources the application, VMs or containers need to perform and then drives the environment to its desired state based on the best use of the underlying compute, storage and network supply.  A state where application workloads performance is assured while efficiency is maximized.

VMTurbo will make the sizing and placement decisions for you so that your time is spent better, and your end users stay happy.

Leave a Reply

Your email address will not be published. Required fields are marked *