Waiting For VMware VVOLs

I've been convinced for the past couple of years that we as an industry have to shift our management paradigm from managing storage as logical volumes (or LUNs) to providing data services to applications or, at the very least, virtual machines. Unfortunately, waiting for VMware's solution to this problem -- Virtual Volumes (VVOLs) -- has started to resemble Waiting for Godot.

In Samuel Beckett's absurdist play, two main characters, Vladimir and Estragon, anxiously await the arrival of Godot -- who, by the way, never arrives. VMware and its hardware partners from its parent company, EMC, to the all-flash startup SolidFire demonstrated VVOL functionality at VMworld 2011 and 2012, but we haven't heard of any significant advancement on VVOLs since.

The storage news at VMworld 2013 was all about VSAN, and VVOLs went without mention in the various keynotes and general sessions. There were a few vendors with technology demonstrations of VVOLs, but they looked to me like reruns of the same technology we had seen a year before.

The biggest news about VVOLs in recent memory was a comment by VMware CEO Pat Gelsinger at this month's EMC World conference: "Perhaps we should have finished VVOLs before VSAN." At the conference, I also heard rumors that VVOLs would be announced at VMworld this August.

VVOLs let block storage vendors provide per-VM data services like snapshots and replication by essentially storing each VM, or VMDK, in its own logical volume. Small organizations can do that today by simply creating a separate LUN and corresponding vSphere datastore for each virtual machine. There's a lot of management involved in creating and managing 50 or 100 LUNs, and organizations quickly run into vSphere's limit of mounting 256 LUNs to a vSphere cluster.

That limit isn't a VMware invention; in fact, it's a limitation of the basic protocols governing block storage, including iSCSI and Fibre Channel. Each connection between a storage initiator, the hypervisor host, and a LUN on a target like a storage array requires a login, and our block storage protocols simply weren't designed to support thousands of logins from one initiator to one target.

The basic concept behind VVOLs is to multiplex block storage requests for multiple LUNs on a single target over one connection. The storage array therefore doesn't have to process and maintain connection logins for each LUN. A single hypervisor host could access more than 256 VVOLs.

Though the concept is pretty simple, the implementation will have more than a few moving parts. VMware and other hypervisor developers -- if they want to take advantage of VMware's work -- will have to create the I/O multiplexor as part of the hypervisor. Array vendors will not only have to build the I/O de-multiplexor function into their arrays, but also plug into a hypervisor management console like vCenter to create VVOLs automatically when an administrator creates a VM or virtual disk.

In addition to supporting the de-multiplexor function, storage arrays running VVOLs will have to be able to support thousands of LUNs and probably tens of thousands of snapshots. Other obsolete storage concepts, such as dedicated snapshot reserves for each LUN, will have to go by the wayside. All this implies that storage arrays will need more CPU and memory to manage the additional metadata required to track all those vVols and snapshots.

Block storage vendors that are serious about virtualized environments, like Pure Storage and Nimble Storage, have been waiting for VVOLs as their per-VM data management solution. In the meantime, a few pioneering vendors -- including Tintri, Virsto, and Maxta -- have gotten the insight into virtual machine data usage they need to provide per-VM snapshots by using NFS or SMB-3 for Hyper-V and running their own snapshots.

The per-VM storage visibility these solutions provide is important not just for snapshots and replication, but also to provide highly granular storage quality of service. With VVOLs, a storage system with QoS features like SolidFire's can give high-priority VMs the storage performance they need when the system is heavily loaded. Today admins have to segregate high-priority VMs into separate data stores, and the storage system can't protect one SQL Server from another if they share that high-priority LUN.

The LUN is a level of abstraction whose time has come and gone. It served an important purpose when SANs supported a handful of servers, but in today's virtualized or even "cloudified" datacenters, it's time to shift to a more granular unit of management. You can address the problem by using a state-of-the-art file storage system for your VMs, but per-VM granularity on block storage will require VVOLs. I, for one, am a bit tired of waiting.

P.S.: In January, I set the over/under date for VVOL availability at Dec. 31, 2015. Only time will tell which bets I need to pay off.

Solidfire, Maxta, and Pure Storage are clients of DeepStorage LLC. EMC has been a DeepStorage client in the past.