UK Records Office: Let's Get Digital

U.K. government and law court records from the 11th Century onward will be freely available to the public starting in 2005, thanks to the country's Freedom of Information Act. However, storing and managing these records poses a major IT challenge for the Public Record Office (PRO), which has set to work on developing a digital archiving system (see FileTek Wins UK Archiving Contract).

Traditionally, the PRO has stored documents only in paper format, filed away in secret to gather dust under the government's 30-year rule. After this period, they could be retrieved -- if anyone still cared or had the patience to locate them.

Under the new Freedom of Information Act, the records office must provide: "A general right of access to information held by public authorities in a timely manner," subject to certain conditions and exemptions [ed. note: of course].

The paper records will pretty much stay as they are, but starting next month, PRO will be accepting electronic documents from government departments, which it will archive digitally, according to Robert Taylor, a PRO spokesman. Welcome to the 21st Century!

"In the past, documents would arrive in paper form, but nowadays it's all emails, or databases, or Microsoft Office documents or even virtual reality documents," Taylor says. By "virtual reality documents" he means 3D models that reconstruct a scene, such as the Bloody Sunday Massacre, as part of a government inquiry."We needed a way to preserve this history in its native format for future generations," says David Ryan, the PRO's head of digital preservation.

The records office hired an outside contractor, Tessella, to help it build the new archive. Tessella picked FileTek Corp., Sun Microsystems Inc. (Nasdaq: SUNW), and Storage Technology Corp. (StorageTek) (NYSE: STK) to supply the various pieces of the puzzle.

FileTek's StorHouse hierarchical storage management (HSM) software will manage the digital files in a repository archived on a StorageTek L20 tape library; Sun will supply the servers. Data on the tape cartridges will be accessed through a Web interface, written by Tessella, that provides access and retrieval of requested information. FileTek's Relational File System (RFS) software will be used with StorHouse to archive hundreds of millions of files.

FileTek beat Sun to the software contract because its HSM product is easier to administer, according to Tessella. "PRO needed something simple, so we went with them over Suns SAM FS product," says Rob Sharp, senior software engineer at Tessella. On a related note, the Met Office, the British weather service, previously awarded a contract for similar software technology to FileTek to support its scientific research.

But back to the records office. If easy, fast access to the data is important, why not go for nearline disk storage? (See NetApp Has 100 NearStore Users, EMC Backs Clariion Into ATA, and FileTek Taps EMC Centera.)"Cost, cost, cost," says Sharp. "The perceived wisdom for 100 Tbytes is that disk is still very expensive... and 90 percent of this stuff probably won’t be accessed, much like the old paper archives." Disk would be overkill for many of these documents, he says.

Right now, the records office has just 2 Tbytes of storage capacity, but it anticipates scaling up to 100 Tbytes over time.

Tessella is hoping to win a support contract on top of the integration work involved in building the archive for PRO. "Inevitably, when they start using this system they will want to refine it and make it more sophisticated," says Sharp. "Files will need to be labeled better as the system grows, for it to stay efficient."

— Jo Maitland, Senior Editor, Byte and Switch