Healthy SAN Manages Change

The phrase change is good” probably wasn’t coined by a SAN administrator.

Michael Passe, storage architect for the Boston-based CareGroup Healthcare System, a network of medical centers and hospitals, found change necessary because of rapid storage growth over the years. Managing that change hasn’t always been a good experience, though.

HealthCare’s SAN grew from less than 1 Tbyte in 1999 to 50 Tbytes today, mostly due to new applications and steady growth of email and documents to support 13,000 employees, along with patient records and other medical data. Passe says he’s gone from six servers to 35, including seven file servers with about 1 TByte of data apiece.

To keep up, Passe has done his share of hardware upgrades. He switched from Brocade Communications Systems Inc. (Nasdaq: BRCD) switches to Cisco Systems Inc. (Nasdaq: CSCO) directors and migrated his data onto new EMC Corp. (NYSE: EMC) Clariions. He’s currently migrating storage from mainframes and an old EMC Symmetrix to a new Symmetrix DMX800. And when he finishes that, he plans an email archive project onto EMC Celera CAS systems he just bought.

To make all those changes manageable for him and CareGroup’s other storage admin (yes, there are just two!), he turned to Onaro Inc.’s SANscreen change management software.“Our SAN presents a major management challenge," Passe says. “Even if we make scheduled changes, it’s almost impossible to go back and verify every LUN, every link. The best we can do is spot check. You might go back the next morning and find out you forgot something.”

SANscreen’s goal is to make sure changes to the SAN don’t cause technical glitches, such as missing paths between hosts and LUNs that traditional storage resource management (SRM) software doesn’t pick up (see Onaro Ships Change Manager, Onaro Lifts Its Cover, and Shai Scharf, CEO & Co-Founder, Onaro).

Passe liked the idea of SANscreen even before he saw the product. In fact, he began discussing SANscreen with Onaro execs before the product began shipping in June 2004 and got it approved in his budget last November.

Passe says he had no problem turning to a new startup for an application that plays a major role in setting up his SAN, because he couldn’t find any alternative. He uses EMC’s ControlCenter (ECC) SRM software but it doesn’t handle change management.

“ECC has not been good at telling me you have a port gone down because of a bad HBA,” he says. “That’s something nobody can notice for days and days and days unless somebody tells you. I get emails instantaneously as soon as it pulls, it tells me, ‘Hey, you’re missing this path for this host.’ ”EMC tried addressing the problem in March with a SAN Advisor module for ECC. EMC pitches SAN Advisor as change management, but Passe says it’s not in the same league as SANscreen (see EMC Tackles Change Management).

“It doesn’t do the path monitoring that Onaro does,” he says. “I told them, ‘You’re not doing change management. It’s not competitive.”

When Passe first installed SANscreen, the application didn’t find many problems, but it’s saved him a lot of headaches since then by notifying him of potential snafus.

“It didn’t find a lot of stuff right up front, but there were ongoing things,” he says. “Our Unix support team, storage team, and Windows team all report to different people. There are always changes going on from the host level that could affect the SAN. Sometimes the Windows team moves servers from one HBA to the next and don’t tell me. The zoning follows, and zones shouldn’t have been moved to that disk. It’s difficult to keep tabs on everything.”

Passe finds SANscreen especially helpful during his current DMX migration. “I’m presenting hundreds of new LUNs to hosts, mirroring off things, copying off new LUNs. It tells me if they’re showing up correctly. It lets you do advance planning. You make a plan, model it, print it out, and then you can validate as you step through the plan. Then you look back in the morning and see if it matches the plan. If there were any deviations, it says ‘This part was missed.’ ”— Dave Raffo, Senior Editor, Byte and Switch