Intelligent Storage Fabrics

There are many benefits to running some software on your Fibre Channel switch. The first and most obvious is eliminating host software while maintaining interoperability among storage arrays. In a typical Fibre Channel environment, some software (a replication program, for example) is run on the array, while other apps (like virtualization) usually run as agents on each host that accesses the SAN.

However, there are also problems with this arrangement: If you're running host agents, you must install software on each host that requires access. The more machines you touch, even with automated distribution software, the greater the chance something will go wrong. And if you're not using automated distribution software, good luck. In addition, with software that runs on your disk array, you commonly (though not always) face the problem of vendor lock-in. If you rely on the software to run your systems, you can't very well change array vendors.

Intelligent Fibre Channel switches bring a new variable into the equation. For software that is typically agent-based, the switch does the job that was once performed by agents, and it does it independent of the operating system, without your having to install anything on your hosts. An appealing solution. Although there's still some vendor lock-in, it's at the less-expensive switch or appliance level. Much of the software available on intelligent Fibre Channel switches is aimed at helping you move painlessly from one array vendor to another.

Reality Sets In

Like so many technologies, when intelligent Fibre Channel switches were first announced they were touted as The Next Big Thing. Now, after years of development, products are on the market and getting attention. Twenty-one percent of the storage and IT pros surveyed by the Enterprise Strategy Group said they have deployed some form of an intelligent storage network, and 16 percent said they plan to go that route within two years. Only 5 percent said they wouldn't consider intelligent storage devices.

One thing to remember: Some applications make sense on the switch, and some don't. One example of a "don't" is full-bore backups. We recommend that you run them off of a server and conserve valuable switch resources. As for the do's, quite a bit of our intelligent fabric review focuses on replication and virtualization. This is because both these applications are easier to implement and more efficient to run on the switch.

Think about it: Replication could be simply implemented as a "port map" or "port mirror" by a switch with a little bit of built-in intelligence. When a write command enters the switch, instead of sending data out the port that the LUN (logical unit number) being written to is on, the switch sends a copy out two ports--one for the primary and another for the replica. Easy, fast, independent of anything but the intelligent switch, and no vendor lock-in except if you must buy a separate replication tool if your switch vendor doesn't support replication.Granted, we're oversimplifying--there's more to it than sending data out two ports, but that's where the intelligence comes in. The destination information in the copied packets must be changed so the replication target knows the writes are intended for it, but with a little bit of processing power and the ability to do replication at a slightly slowed rate, this is all feasible in the switch.

The same is true of virtualization, which makes the switch into a proxy for the storage. A mapping table is made of virtual LUNs and blocks on them, corresponding to actual physical disk locations. This mapping table is maintained, and when it's time for a write to occur, the operating system requests a write to the virtual LUN. Some other entity uses the mapping table to translate that write to the correct physical location in the SAN. Typically, that other entity is the host agent, a specialized driver that must be written and maintained for each supported operating system and deployed on each host that requires access to the SAN or an inline appliance. By moving virtualization software onto the switch, the host agent is removed from the equation. Alternatively, the inline appliance--and the latency it introduces--is removed. Although processing must still occur, it happens on the switch, making one stop for your data instead of two. The switch is getting standard Fibre Channel packets, so no matter which operating system made the request, support for all operating systems is automatically provided. And because the software running on the switch does the mapping from virtual locations to actual physical locations, you don't need an agent on the host. All in all, it's more adaptable and stable. And as a final bonus, switches tend to be designed for high availability because of their location in the network. This isn't true of software and rarely true of inline appliances.

So why isn't everyone rushing to install intelligent switching devices? Early failures did keep buyers away--latency and storage networks don't mix, and it's difficult for jaded IT pros to believe you can put intelligence into the data path without introducing latency. Then there's the obvious: The current setup works, so why in the world would we change it?

This market is mature enough that there are working products and young enough that adoption isn't widespread. All the major switch vendors and several smaller ones offer products that tackle latency problems. The challenge remains user education and public relations. There has been a reluctance to do anything that might interrupt the all-important SAN, and because the switch is the "nerve center" of the SAN, customers are leery about changing switch functionality without a darn good reason.Here's another reason to be wary: These smart switches eat up ports in your SAN. With the possible exception of Cisco's--depending on your deployment scheme--every product we've tested or been briefed on requires ports. This might cause intelligent switch vendors more grief over the long term than any other concern. In more than one product, you must use multiple ports on your switch or all of the data on your SAN will travel through a single 2-Gbps link. And in at least one case, or if you're not careful in a multiswitch fabric--you'll end up routing everything through a single port on a single switch! More on these kinks and how each vendor handles them in our product review.

Another significant reason enterprises have been reluctant to implement intelligent Fibre Channel switching is the dearth of applications. Switch platforms are necessary, but equally important are apps to run on them. This year we've seen many more third-party software vendors getting on board. But some switches run only internally developed applications, and those are generally the most feature-rich.

But as vendors with programmable interfaces bring in more third-party developers, we expect the market to change. For example, Cisco Systems and McData are working with third parties to bring products into their switches that would normally take years to develop in-house. Because these are separate development efforts with dedicated teams, it's less of a drain on the switch vendor's resources and more applications can be developed in parallel.

Money Matters

If you own a Fibre Channel SAN, the cost of adding intelligence to the switch will vary wildly depending on your environment and the vendor you choose. If you can get away with just buying blades or external devices for an existing infrastructure, your storage may get smart inexpensively, but consider these questions: Do you have space in your chassis for the blades? Do you have open ports to use for the processing connections, and how many connections will you need to provide the bandwidth required of your storage-area network? Take virtualization. In the old model, you could depend on having the full bandwidth of your switch available when writing to a drive. The math didn't change because virtualization was introduced--you still wrote X gigabits of data and assumed that if your storage targets had enough bandwidth you were set.With intelligent switching, you must have enough bandwidth from the host to the intelligence facility, and then enough from the intelligence to the disk. In some instances, this setup will be the same as what you're accustomed to, but in many cases the "intelligence facility" will be on another switch somewhere in the SAN. It may require multiple ports between the two switches to ensure you have sufficient bandwidth to handle all of your SAN traffic.

This isn't a major stumbling block, but you should be aware of it going into your deployment. If you normally peak at 12 Gbps through a particular segment of your SAN, obviously a 2-Gbps link between switches that all traffic is routed through isn't going to cut it. Make sure you have enough ports to cover the load if your intelligence won't be in your core switch, and count the cost of buying more ports if they're required.

Overall, this market is mature enough that it's safe to begin implementations, but some kinks are still being ironed out. For example, even though some standards are in place (and more are coming), most vendors have vastly different architectures. But software vendors won't tolerate developing applications for several different platforms for long--eventually one or two intelligence architectures will rise to the top, and other vendors will convert or die.

We expect that Cisco's basic architecture will win, with the Brocade-McData model also remaining viable (see "Different Approaches, Same Goal"). Everyone else will have to follow suit. This uncertainty shouldn't stop you from adopting an intelligent switching fabric where it makes sense, but be aware that several vendors have already disappeared and more are likely to in the coming years. Caveat emptor, as always.

Application Availability: The following software vendors have or are working on software for intelligent switches: EMC (with its Invista for Cisco's MDS 9000), IBM (SAN Volume Control for MDS 9000, Incipient, Revivio, StorAge and Veritas (owned by Symantec). Veritas representatives wouldn't speak to us about the company's support for intelligent platforms, but it has been shipping its Veritas Storage Foundation for Networks on the Cisco 9000 series since the end of 2003.Bad Habits: We find it interesting--and more than a bit disturbing--that NAS and iSCSI seem to be following the path Fibre Channel took, where array vendors provide much or all the software and there's no centralized management for heterogeneous environments. Virtualizing data over an iSCSI network requires host agents, one of a very few centralized products or a single-vendor setup. Seeing as there is no "iSCSI switch," let's hope a new market emerges to centralize this type of functionality.

Moving storage intelligence from hosts to switches isn't a new idea--vendors have been talking the talk for years. But only recently have enough vendors walked the walk for us to put these bad boys to the test. In "Storage Switches Get Smart," we take a look at what makes a Fibre Channel fabric intelligent, what you can gain by moving software that conventionally runs on the array to the switch, and why some storage pros are taking it slow (hint: latency fears laced with sticker shock).

We tested smart SAN switches from Cisco Systems, Maranti Networks, Maxxan Systems and Troika Networks in our Green Bay, Wis., Real-World Labs®. Although we didn't choose a winner because the architectures are dissimilar, we tried our best to overload them to see whether latency concerns are valid. All performed very well, but SAN interoperability is not yet within our grasp. See results of our tests in "Right Place, Right Time", along with a discussion of each device's strengths and weaknesses.