iSCSI SANs

Another advantage of iSCSI compared with Fibre Channel is the distance over which it can be used. Fibre Channel is limited in this capacity, but iSCSI uses TCP/IP, which means it can be set up for use at great distance over the Internet for very light storage traffic or over leased lines without the need for the expensive optical gear Fibre Channel requires.

The Need for Speed

iSCSI does have some drawbacks. Current iSCSI technology is limited to 1 Gbps--the speed of Gigabit Ethernet--while Fibre Channel solutions have a maximum transfer rate of 2 Gbps (4-Gigabit FC is on the immediate horizon). We expect that iSCSI eventually will be deployed on 10 Gigabit Ethernet, but for now, that's not possible--there isn't a TCP/IP off-load engine capable of handling those speeds.

In three to five years, when 10 Gigabit Ethernet becomes more prevalent, you'll see iSCSI follow suit. But if your current requirements for responsiveness and backup speed can't be met at 1 Gbps, iSCSI is not the technology for you right now.

Similarly, if you've already invested a great deal in Fibre Channel technology, you may want to think seriously about making a considerable iSCSI investment. There is a definite case for using iSCSI in a limited capacity for distance applications and in less speed-critical situations. It can also be an inexpensive way to move forward if you are currently at capacity on your Fibre Channel switches.iSCSI Concerns

Remember this: You shouldn't expect to run your SAN network on the same generalized TCP/IP network that carries data between servers and users. Storage is simply too important to be on the generalized data network--it must be segregated onto separate VLANs or an entirely separate switched network. (There are exceptions to that rule, as in the case of a small workgroup using iSCSI for limited access to a main storage device or accessing storage remotely.)

Another consideration: iSCSI places a large burden on a server's CPU. iSCSI is essentially a wrapper around standard SCSI commands. Those commands must be received in order, and each command or data packet must be wrapped (or unwrapped). This creates a huge amount of TCP/IP work that the standard NIC card passes off to the CPU.

The solution is a TCP Off-load Engine (TOE) or an iSCSI Host Bus Adapter (HBA). These specialized cards perform all the TCP/IP overhead processing with on-board processors instead of passing it to the CPU the way standard NIC cards do.

TOE cards and iSCSI HBAs each have advantages and drawbacks. Whereas iSCSI HBAs deal only with iSCSI traffic, TOE cards can handle all standard Ethernet traffic. However, iSCSI HBAs use fewer main CPUs than do TOE cards for iSCSI operations, and they generally cost a few hundred dollars less than TOEs.In our test of TOE cards and iSCSI HBAs, (see "Don't Sink Your IP SAN" at www.nwc.com/1409/ 1409f3), we found that if a TOE card or HBA has only one target, it's unlikely to achieve a data rate of 100 Mbps because parallelism in the off-load processors prevent it.

Other factors in the TOE-versus-iSCSI-card debate include whether you intend to repurpose the card and whether the CPU on the server could benefit from the off-loading of other non-iSCSI TCP/IP processes.

Making a Choice

If these iSCSI drawbacks make you think twice, remember that Fibre Channel still remains the dominant SAN technology, and FC vendors have made great strides in their battle against almost legendary compatibility problems (see "High on Fibre" at www.nwc.com/1325/1325f5.html). In addition, much progress has been made in managing Fibre Channel hardware, though there is still much to do. For example, software tools that manage hardware from multiple vendors can only handle about 50 percent of the hardware out there. Interfaces and terminology from vendor to vendor still vary widely, especially regarding the names of specific kinds of ports. Bottom line: The Fibre Channel industry can't yet manage the number the ports on the outside of the chassis in the same manner.

If you've decided to bring iSCSI to your SAN, what's the best way to proceed? Start with the storage itself. All the usual considerations for a storage array come into play: reliability, speed, capacity, expandability and price.Next, consider what type of hard disk drive you'll need. There's no such thing as an Ethernet hard disk drive. All drives are a form of SCSI, Fibre Channel or ATA (AT Attachment). Thankfully, some companies, such as IBM, LSI Logic and Adaptec's Eurologic Systems, make enclosures that act as iSCSI targets--this eliminates the need to buy a translative device. If you buy the storage device with its native interface, however, you must attach it to a storage switch from Cisco Systems, McData (Nishan Systems), Crossroads Networks or another translative device vendor, which creates an extra step. Still, if you already have these drives, that's the way to go--use the brand of Ethernet switch you're used to. If you're a Cisco shop, use Cisco. If you're a Nortel shop, use Nortel.

To the switch, an iSCSI packet is just another TCP/IP packet. Network usage levels are determined the same way as they are on the rest of your Ethernet network. At the server or, in iSCSI terms, the initiator, you simply have to choose a TOE card or an iSCSI HBA. iSCSI HBAs generally come with their own software initiator, while TOE cards use a third-party initiator to work. Microsoft and the open-source community offer iSCSI initiators free.

Steven J. Schuchart Jr. covers storage and servers for Network Computing. Write to him at [email protected].

Post a comment or question on this story.

1) Run your iSCSI SAN on a separate network, not on the generalized data network.

2) You'll need TOE or iSCSI HBA acceleration to avoid excessive CPU utilization.3) Consider parallelism when deploying TOE/iSCSI HBAs.

4) Remember that Ethernet switch usage is the same for iSCSI as for any other Ethernet application.

For details and prices on specific makes and models, use our Interactive Buyer's Guide charts.

Web LinksInteractive Buyer's Guide charts on iSCSI SANs

Storage white papers

Storage research reports

Storage books