Applied Discovery Deploys 8-Gig FC to Handle Data Boom

For Applied Discovery, which helps corporations prepare for trials by cataloguing their electronic data, business has been very good lately. The company has benefited from recent regulations, such as the U.S. Federal Rules of Civil Procedure, that force companies to catalogue their corporate information and then maintain it for several years. The growth also meant that the company was straining its IT infrastructure, a situation that led to the building of a new data center and the adoption of an emerging high-speed network interface, 8-Gbit/s Fibre Channel.

Applied Discovery, which has approximately 200 employees, was founded in 1998; it is now a division of LexisNexis, a workflow solutions provider. Large law firms and legal teams in Fortune 500 companies use Applied Discovery's e-discovery products and services to pull out relevant data from all of their corporate information whenever legal issues arise. For instance, the company will take all of a client's emails, whittle them down to those relevant to a case, and deliver them to the customer.

Business had been booming, but that good fortune created new challenges for the company. "The volume of data that companies have to look through grew from hundreds of gigabytes a few years ago to 3 terabytes to 4 terabytes now," says Keith E. Moore, director of technology services at Applied Discovery. As a result, the company needed additional computing and storage capacity as well as more flexibility in how it allocated computing resources.

In response, the information services company opened a new data center at the end of 2007. It now has 1,000 servers, largely HP and Sun systems, broken into data management systems, index engines, and raw processing systems. The legal service company's storage area network has more than 1 PB of information.

As it was moving into its new data center, Applied Discovery spent a few months reviewing its traffic patterns and trying to pinpoint the source of its network or system bottlenecks. "The Fibre Channel links in our data center were becoming a pressure point," says Moore. Hundreds of the company's servers were connecting to network switches sitting in front of its storage systems, and the volume of data moving among the switches, servers, and storage systems would occasionally overload the Fibre Channel connections, which had been in use for about half a dozen years.Applied Discovery compared Cisco switches, which had been used in its data center for those six years, and Brocade devices, which had been used by a couple of departments for special applications. "We saw better performance -- closer to full line speed -- on the Brocade systems' ports than with the Cisco product," says Moore.

The Cisco switches worked well at low speeds, but whenever the company moved large files, latency problems arose and disk queues became longer. Applied Discovery thought that the Cisco system's backplane may have been getting overloaded. So the company decided to purchase Brocade switches in 2007, justifying the purchase, which cost a few hundred thousand dollars, by the efficiencies gained from the better network performance.

However, its move to 8-Gbit/s FC ran into a roadblock. "Our switches were ready for 8 Gig Fiber Channel, but none of our storage vendors supported the higher speed yet," Moore says.

As a result, one of the first places where the company adopted 8-Gbit/s Fibre Channel was between a couple of Brocade switches. The switches provide redundancy and support functions, such as encrypting data before it is sent to a storage system.

With virtualization becoming more popular, more companies are front-ending their servers, or connecting a lot of servers to a switch to consolidate traffic and feed it to a storage network, so their storage and bandwidth requirements have been growing. In response, storage vendors have slowly added 8-Gbit/s Fibre Channel support to their systems.Hitachi was the one of the first storage vendors to support the new interfaces, so Applied Discovery purchased its product in the summer of 2008. "We ran the 8-Gig Fibre Channel links in the lab for several months before deploying them," Moore says. "We wanted to make sure that there were no issues with them." The company migrated over to the new system in February 2009 and did not encounter any problems.

However, there could be some potential issues for other companies. Applied Discovery's system works with host bus adapters from only one vendor, QLogic. If a company has a few vendors' products, compatibility and performance issues could arise. Unless all of the HBAs support 8-Gbit/s Fibre Channel, these systems will fall back to slower transmission speeds.

Now that it has 8-Gbit/s Fibre Channel up and running, what's next for Applied Discovery? "We have no need for 16-Gbit/s Fibre Channel at this time, but probably will at least be thinking about it in another 18 to 24 months," Moore says.

InformationWeek Analytics has published an independent analysis of the challenges around enterprise storage. Download the report here (registration required).