Dell's Into Primary Dedupe, Who's Next?

Dell, with their announced acquisition of Ocarina Networks, put themselves squarely in the deduplication market joining both NetApp, Oracle (SUN) and EMC as storage system companies offering some form of primary storage deduplication. While it remains to be seen what Dell does with the Ocarina technology, it clearly sends a signal to other storage system vendors that primary deduplication is going to be a required capability just like snapshots are today.

Dell could do a few things with the Ocarina product. It is safe to assume that they will use the current Ocarina product as is to offer deduplication to their entire customer base. The Ocarina product was originally sold as an appliance that can be added to nearly any existing storage infrastructure. It seems reasonable to assume that Dell will also begin to look at integrating the technology into their EqualLogic family of products. It will be interesting to see if and how Dell uses the Ocarina product with the products that it OEMs from EMC. They could use it to differentiate themselves from the throng of EMC resellers. All of that is conjecture right now, and we will have to wait and see what Dell does. More interesting for the moment is what exactly does the rest of the storage community do?

Clearly Dell joins the growing list of storage systems manufacturers endorsing deduplication of primary storage as an important element in the product offering. The challenge that the remaining storage manufacturers face is that if they don't have something already ready to implement, they are running out of time to develop their own. Unless they have a deduplication development effort underway, there are two choices: buy a deduplication technology or OEM one. From the buying perspective there are fewer companies left to purchase. While it is safe to assume that another company may come along to offer deduplication as an OEM service, clearly the choices are limited and becoming more so. Also as we move out of the larger tier of storage vendors there is less money available to buy a company offering primary storage deduplication technology.

From an OEM perspective there are products that either are file systems or are add-ons to file systems that provide deduplication capabilities. These vary in scope and efficiency but for file based customers they may have an appeal. The time to market may also be faster since there is less integration work. The obvious limitation here is that these are file system-based solutions, and as a result the deduplication functionality is limited to the file system that it controls. Using this type of solution may mean a bit of a feature burn for some storage manufacturers, turning off much of their intelligence and leveraging what the file system gives you. Depending on the level of data services the storage system provides will determine if this is a net gain or loss.

The other option for storage system manufacturers is to integrate at an API set level deduplication hardware or software. While this will take extra development time on the part of the storage systems integrators, each should not be a half-year project. Anecdotal evidence suggests maybe a few weeks. The advantage of this approach is then you have a native interaction between the storage controller and its data services and the deduplication engine. This should mean that all the investment that the storage system manufacturer made in providing data services like snapshots, thin provisioning and automated tiering could continue to be controlled by the storage system's manufacturer. It also means that the storage system manufacturer should be able to offer deduplication across all storage types NAS or block. These models each have their own distinct value. No matter what choice the storage manufacturer makes, it is becoming clear that primary storage deduplication is quickly moving from something nice-to-have category to being a must-have.