Dedupe Everywhere Changes Economics

With the new versions of Backup Exec and NetBackup due Monday, Symantec's finally joined CommVault, IBM, Atempo and most of their other backup software competitors by integrating data deduplication into their mainline backup software. Now that you can dedupe data your media server of your choice we have to ask the question "Where is the most cost effective place to dedupe backup data?"

For the past few years the conventional wisdom was to point your backup software at a deduping disk target like those from Data Domain, Exagrid, Quantum, Sepaton and FalconStor. Compared to replacing your backup software with a source deduping product like Asigra or Avamar integrating a backup target, especially a VTL, into your existing backup process is relatively painless and they can suck up data at a prodigious rate (up to 5TB/hr for a Data Domain DD880).

Deduping targets are probably still the best solution for enterprise data centers where users around the world keep servers busy around the clock minimizing both backup windows and host CPU cycles available for backup tasks like source deduplication. Smaller shops typically run a single shift leaving a 12 hour nightly backup window when servers, even virtualized servers, have cycles to spare.  The questions then become: How much could a shop with a few terabytes to backup save? And is media center dedupe fast enough?

An entry level deduping disk target like a Quantum DXi 3500 or Exagrid EX4000, both of which have 4TB of usable disk space, costs $40-50,000 which could be a significant chunk of an SME's total hardware budget for the year. Integrating the appliance into the network and updating the 20-100 backup jobs a company this size would have should take less than a day.

If that company was a Backup Exec user, as many are, they could add 4TB of disk to their backup server and add the $1995 data deduplication option instead. A Dell MD1000 SAS JBOD with PERC 6/I RAID controller and 15 500GB drives is about $7200. Even if they decided to have their favorite consultant or VAR build a new Xeon 5500 server, with beaucoup memory as Symantec's dedupe needs, about 1GB of memory for each TB of disk and other Backup Exec features like granular recovery of Exchange mailboxes are memory intensive themselves, and total cost will be on the order of half that of a deduping target. Other vendors including CommVault charge for dedupe on a capacity basis but not enough to change the basic economics.The other question is how fast do backup apps actually dedupe data? The proof of the pudding is in the eating, and we're looking forward to getting deduping backup software from Symantec and others in the lab so we can generate some real-world metrics and best practice recommendations.

Note: We at DeepStorage.net do work for Symantec occasionally.