VMware VSAN: Reality Vs. Hype

We are enjoying a period of intense innovation in storage, one that has questioned long-held beliefs about how data is delivered to servers. A prime driver in this innovation is the cloud, where masses of inexpensive gear are deployed virtually to serve the customers’ need. One issue the cloud raised in its early evolution is the problem of getting adequate storage performance to virtual instances. Hard disk arrays and the networking available limited instances to much lower IO rates than they would see from a local (DASD) drive.

The solution to this is the addition of local temporary “instance storage” in the form of a hard drive, or more likely a solid-state drive added to each server. This certainly does the job, though at the loss of some security and “statelessness,” which manifests in longer setup/tear-down times, a more comprehensive erasure process on the instance storage, and a more complicated replication system for data protection.

Enter the VMware Virtual SAN. As a logical step up from a local drive, VSAN extends the concept of local instance storage to a shareable storage unit in each server, where the data can be accessed by other servers over a LAN.  Proponents of the idea even discuss replacing networked storage appliances with this approach, putting several drives in each server.

Let’s look at the value proposition for VSAN and see if reality matches hype. The touted benefits of VSAN include:

The performance question doesn’t hold up well under scrutiny: VSANs with 10 GbE links can service as many as four SSDs, and RDMA links at 25 GbE are due in 2015 to boost that capability considerably. However, competing storage appliances have the same interfaces, and typically go up to 40 GbE today. By mid-2016, this will be 100 GbE. 

To get a better sense of the network issue, we have to look at the count of network connections in the system. This is a fairly complicated model, since the typical two LAN connections on a server get their workload restructured significantly.  With a traditional structure of lots of storage behind an expensive head node, the equation roundly favors the VSAN in terms of total IO network bandwidth. The problem is, we are evolving towards a smaller drive count per storage appliance, and more, cheaper appliances, which nullifies this advantage.

Now let's examine the cost argument for VSAN. Surely, not having storage appliances is a cheaper solution, right? That’s true if a server is a large box with a bunch of drive slots; these can hold a meaningful VSAN module of drives. Again, though, the server is being redefined. We have units that are no-frills “white-box” designs appearing even from mainstream vendors such as HP. These can’t hold more than one or two drives and server packages will not be able to contain a VSAN.  But they are cheap!

Meanwhile, the storage appliance folks are doing the same tricks to slim down and cheapen their boxes. No more gold-anodized frames and expensive drive caddies. No more fiendishly expensive disk drives. Commodity is king! The bottom line is that servers that hold VSAN drives are going to be much more expensive than the high-volume, no-frills white boxes.  So we can scratch VSAN cost savings as a myth, too.

At the same time, scalability also is a VSAN myth. Typical  data center installations of systems don’t scale linearly. Some need much more storage growth than server growth, or vice versa. The boot storm issue is actually more of an issue with how images are managed, and a far more elegant solution is evolving in the form of container virtualization, which nullifies having to set up OS images for each instance.

That leaves unified management. Proponents of VSAN talk about the wars between storage and server teams in IT. There is certainly competition, but a “war?" Any CIO with that problem needs to take action other than changing to VSANs. Unifying management is like getting a single dashboard to control everything. The market will solve this problem as software-defined infrastructure takes control.

This discussion has repercussions in the so-called converged and hyperconverged systems arena. Essentially, the same questions apply, and the same answers. This whole area is a mashup of hype and FUD, along with some technical genius.

Tread carefully, since the sea change in storage is in its early days.  White boxes are just starting the race for the bottom on price. Traditional storage vendors feel great exposure to these new solutions. Small, very smart appliances change the cost and performance profile of storage tremendously, while compression and deduplication will change how storage is provided. With 10U bulk drives and 84-drive 4U cabinets later in 2015, compression will yield a petabyte per 1U of rack space!