Gigabit Ethernet NICs Toe the Line

Throughput gains for NICs with TOEs can be dramatic. 3Com's 3C996-T 10/100/1000 PCI-X Server NIC with basic checksum processing and interrupt coalescing can sustain bidirectional throughput of 300 Mbps to 900 Mbps. A NIC capable of higher-level TCP/IP processing, such as Alacritech's 1000x1 Server and Storage Accelerator, can reach bidirectional throughput speeds of 1,500 Mbps to 1,800 Mbps while reducing host CPU utilization (see "Alacritech's 1000x1 Server and Storage Accelerator Is a SLIC NIC").

Toe-to-Toe Comparison

TOEs are distinguished by their full or partial off-load solutions. Full off-load solutions, such as Adaptec's ANA-7711 TCP/IP Offload Adapter, remove all TCP/IP protocol processing from the host to the NIC. If your environment has intermittent connection establishment and termination and is prone to dropped packets, a full off-load solution would be best. A partial off-load solution, such as Alacritech's 1000x1, automatically off-loads the data transmission/reception information or data path to the NIC while the host TCP/IP stack retains the responsibility for connection establishment and termination and error handling. If your network uses fiber optic media, has little problem with dropped packets and maintains connections for long periods, look to a partial off-load solution.

TOEs can be implemented in discrete component architecture using off-the-shelf computer components or in a specialized ASIC. Discrete component TOEs are constructed using a circuit board and off-the-shelf computer parts--a network processor or microprocessor, firmware, memory, data transfer bus, a RTOS (real-time operating system) and a PHY/ MAC interface. The protocol processing usually done by the host CPU is off-loaded to the embedded TCP/IP protocol stack in the RTOS.

The advantage of building board-level solutions for TOEs is flexibility. Individual compon- ents can be changed and firmware can be updated in the ROM chip. You can modify the TCP/IP stack with a firmware upgrade and tune it for special processing environments.The alternative to building a TOE is to integrate hardware and software into customized silicon. This solution comes in two flavors: FPGA (field-programmable gate arrays) and ASICs. Like firmware-based solutions, FPGA solutions are flexible because software can be changed dynamically. But high cost limits FPGA solutions to initial implementations and product development. Once all the bugs are worked out and the manufacturing process is set, TOEs are implemented as ASIC solutions, which cost less in volume production.

Although the same TCP/IP off-load processes run on both implementations, the ASIC solution has processors and memory on-chip. As a result, ASIC TOEs boast better performance than that of their firmware counterparts. However, if you need a customized solution, you may want a solution that can be upgraded with firmware.

Taking it to the Streets

Last year Dell's PowerEdge 2650 server shipped with an Intel Gigabit controller as standard. This year, an Intel Pro/1000 MT Gigabit Server Adapter sold for $139.95. The dual-port version sold for $189.95, and a desktop version sold for $54.95. This autosensing 10/100/1,000-Mbps NIC gives you an example of how the cost has come down for gigabit connectivity over Category 5 cabling.

Although the PowerEdge and the 1000 MT do not have full TOEs, they provide TCP segmentation off-load in hardware. But the more off-load features provided, the greater the cost. For example, Alacritech's partial off-load solution (the 1000x1) for both TCP/IP and iSCSI using an ASIC sells for about $1,000, and Adaptec's full off-load solution (ANA-7711) will cost more when it ships later this year.Not only does using NICs with TOEs provide immediate benefits in server performance, in the long run the server resource savings may reduce the number of servers required to provide adequate QoS (Quality of Service).

Server administrators often choose between scaling up in CPU resources or scaling out in the number of CPUs to satisfy computing demands. With TOEs, you can off-load TCP/IP processing and reserve more CPU power for application processing. The use of TOEs may also lead to a flattening of the network and convergence.

Traditional data centers use various network layers of routing and switching to optimize and accelerate server performance. For example, one layer may aggregate slower networks into gigabit pipes. Another layer may accelerate performance by leveraging dedicated server-side cache appliances and load-balancing technology. TCP/IP off-load technology lets servers fully use gigabit pipes without an aggregation layer. They may also reduce the need to accelerate server performance. In addition, TOEs support the proposed iSCSI standard and iSCSI uses more CPU resources than TCP/IP processing because of the additional packing and unpacking of block SCSI frames.

When 10 Gigabit Ethernet is standardized later this year, block-level data can be transported over TCP/IP faster than Fiber Channel's 2-Mbps speed. This will give enterprises an alternative to running a separate storage network and converge application and storage solutions over TCP/IP.

Sean Doherty is a technology editor and lawyer based at our Syracuse University Real-World Labs®. A former project manager and IT engineer at Syracuse University, he helped develop centrally supported applications and storage systems. Send your comments on this article to him at [email protected].The TCP/IP off-load process generally refers to the capability of engaging protocol processing at OSI Layers 3 and 4. Many of the NICs in our survey use Gigabit Ethernet controllers and hardware acceleration to process TCP, IP and UDP checksums and TCP segmentation at the MAC (Media Access Control) and physical (PHY) layer. But the full range of TCP/IP protocol processing can be separated into four processes: connection establishment, data transmission/reception, connection termination and error handling.