A National Archiving Challenge

By Paul Travis, January 2, 2009 1:00 PM

Over the next few days, most people will be reflecting on 2008 and making plans for the next year. For storage administrators and IT managers, the year ahead will feature some difficult challenges -- a bad economy, tight or shrinking budgets, and more data that needs to be stored, backed up, protected, and archived. But as you ring in the New Year, take a moment and give a toast to the folks over at the National Archives and Records Administration, who face a unique challenge.

On Jan. 20, NARA will take charge of a mountain of paper documents and around 140 TB of data from the outgoing Bush administration, more than 50 times the amount of electronic information it received from the previous Clinton administration. The job is to sort, index, protect, preserve, and provide access to all paper and electronic records created by the federal government during the past eight years.

Under the law, the federal government owns and controls all presidential and vice-presidential records, and the National Archives is legally responsible for "the custody, control and preservation" of the records, according to a recent story in The New York Times. That includes "top-secret e-mail tracing plans for the Iraq war as well as scenes from the likes of Barney Cam 2008, a White House video featuring the first pet," the newspaper reported.

The National Archives has enacted an "emergency plan" to handle the records. The agency plans to take over parts of the White House storage system and freeze its contents on Jan. 20. After that, archivists will analyze the records and then move them into a new a repository for digital data, the story said.There are questions as to whether the new system can handle the job. The budget was cut in the middle of development and deployment, it is behind schedule, and it isn't clear that it can handle all of the data formats that the administration used for communications. Additional complications include lots of missing emails, high-level officials who think they can decide what will be included in the archives and what won't be included, and a host of lawsuits seeking access to many of the administration's private communications.

NARA's system uses database technology from Oracle Corp. (Nasdaq: ORCL) and EMC Corp. (NYSE: EMC)'s Documentum for records management, search technology, and a Web-based front end, according to a story in Computerworld. It includes a storage system from Hitachi Ltd. (NYSE: HIT; Paris: PHA) that blends servers from EMC, Hitachi, and Sun Microsystems Inc. (Nasdaq: JAVA), as well as the Hitachi Content Archive Platform, which automatically indexes records as they enter the system, enabling immediate search capability, the story said.

Most storage admins understand the challenges NARA faces. And it isn't about storage. The Times quotes Paul Brachfeld, the archives inspector general, who raised questions about how quickly the records will be made available to Congress, the courts, and researchers. "The electronic records archives system may be able to take in a tremendous amount of e-mail and other records," he said. "But just because you ingest the data does not mean that people can locate, identify, recover and use the records they need."

That's the bottom line, isn't it? The job of storage administrators is never just about storing data. Today, the job is about all of those other things: locating, identifying, recovering, and using. And those tasks will be even more important in 2009. Most of you will be asked to do all of them and much more in the next year, probably with smaller budgets and fewer personnel. It means that next year will be one of the most challenging that we've seen in a long time. But cheer up -- you could be working for the National Archives. Happy New Year to you and yours. I'll talk to you again in 2009.