Tape Rules for Long-Term Storage

When I talk to users about storing data for the long term, I frequently get pushback when I suggest that tape might be a good fit with their requirements. A common concern is that the data will have to be periodically migrated from old tapes to new. With IBM's recent announcement that LTO-6 drives will soon be available, it seems time to examine just how often you'd have to migrate data you want to store for 25 years using common storage technologies.

Data migration is just one requirement to consider when choosing a long-term storage archive. There are also storage APIs, acquisition costs, power consumption and--most significantly--data integrity. That said, data migration will affect the cost and reliability of your long-term archive. Note that the process I'm proposing here is appropriate for organizations with hundreds of terabytes or more of data. If you have 20 to 50 TB of archive data, you probably can't justify the investment in a tape library and should stick to a disk or cloud system.

If you choose a standard disk system like an EMC VNX or NetApp FAS for long-term storage, you'll have to migrate your data to a new system roughly every five years. While you could probably run an enterprise NAS seven or eight years, most vendors will either significantly boost maintenance costs after the fifth year or simply refuse to write a new service contract on a 6-year-old system. Rather than store data on an old system without support, most organizations will migrate at the five-year mark.

Tape users can probably go 10 years between data migrations. Tape vendors say data on tape is stable for 20 years, so tape should be readable after just 10. Because I'm not the trusting type, I'll write each file in my archive to three separate tapes. Two of these will stay in the tape library and the third will be sent off-site to protect against fire, flood or other disasters.

For added insurance, I'll buy a tape library that can automatically verify tapes for me, such as those from Spectra Logic or Quantum. My tape library will load each tape once every two years and verify that the data can still be read. Should a tape read fail, I'll get the data from the second tape and make an immediate copy to a new tape.

I'll also get 10 years or more out of my tape library because tape library vendors are willing to write service contracts for significantly longer than disk storage vendors. Spectra Logic, for example, will write a maintenance contract for any library it can still get all the parts for. Many organizations are running 10-year-old libraries even after having upgraded from LTO-1 to LTO-5 drives.

While tape library vendors will write maintenance contracts for older tape libraries, all is not sweetness and light. They will, for example, assign the library's warranty expiration date to all the components in the library. If you add new tape drives, or upgrade your LTO-4 drives to LTO-6 drives, the new tape drives will essentially have no warranty coverage as you are putting them in a library whose warranty ran out years ago.

It's at this point in the argument when someone usually says, "But tape vendors come up with a new format every three years, and I'll have to migrate my data to stay current." Not so. The people making that argument don't migrate data from one disk drive to another just because newer, bigger disk drives are on the market. In the same vein, many of us have recalled boxes of old backup tapes from Iron Mountain and discovered that we no longer have the right tape drive to read those old tapes. However, this isn't necessarily an argument against tape, just an argument for good management.

Since 2001 or so, the tape market has been dominated by the LTO Ultrium format. Each generation of LTO tape drives can read tapes back two generations, so the LTO-6 drive you buy next year will be able to read LTO-4 and LTO-5 tapes. Hewlett-Packard and IBM still make LTO-3 drives, so any LTO tape ever written can be read with tape drives you can still buy new. (This is more than I can say about the old 8 mm tapes many of you have in storage.) If you partition your tape library and keep LTO-3 and LTO-5 drives online, there's no need to migrate data from any old LTO tape just because of format obsolescence.

Of course, all of this assumes that you're using good archiving software that can restack data from five LTO-1 tapes to one LTO-5 tape when the time comes. Software is, in fact, the key to a long-term archive, but if you have enough data to require a good-size tape library, you should be able to migrate your data less often using tape than traditional disk systems.