EEMBC Develops Networking Benchmark Techniques

EEMBC is hoping to have a bigger impact on the networking design process with the release of version 2.0 of its networking benchmarking specifications.

EEMBC, which is best known for its processor benchmarks, has offered a basic networking benchmark for some time that performed basic packet checking operations, said Marcus Levy, president and founder of EEMBC. Now, after gaining quite a bit of interest, EEMBC has added a host of new benchmarks that will allow designers to evaluate quality of service (QoS), network address translation (NAT), TCP, and IP performance in networking designs.

"The new version provides more real-world benchmarks," Levy said. "These benchmarks are built on actual algorithms defined by the International Engineering Task Force (IETF)."

The new benchmarks include a QoS benchmark, a NAT benchmark, an open shortest path first (OSPF) benchmark, and an IP packet check benchmark. The QoS benchmark simulates the processing undertaken by bandwidth management software used to shape traffic flows to meet QoS requirements. The system paces the delivery of the packets to the desired speed, based on a set of predefined rules. This shaping is achieved via the use of a variant of the weighted fair queuing (WFQ) algorithm. Random Early Detection (RED) queue management is also supported to provide flow control.

The OSPF benchmark implements the Dijkstra shortest path first algorithm. This algorithm finds the shortest path from a router to all other routers. Instead of building a predefined route, which is typically done in an OSPF system, EEMBC's benchmark builds the routing tables dynamically. The benchmark then repeatedly walks the list that is used to hold the nodes. Consequently, a processor's load-use latency and its ability to handle frequent control transfer instructions (CTI) operations are measured by the benchmark. NAT Checking

The NAT benchmark focuses on the handling of egress packets. When a packet arrives, initial processing ascertains what action, if any, needs to be undertaken.

The NetBSD NAT benchmark implementation uses a 128-entry hash table to hold information about current connections. By using the source address, destination address, protocol, and ports (if applicable) of the packet, the system computes an offset into the hash table. If this entry in the hash table relates to the current packet, the packet belongs to a connection that is already established and the packet processing is undertaken as dictated by the NAT table entry.

If the packet doesn't belong to a current connection, the list of NAT rules are searched to ascertain if a rule exists for the packet handling. If a rule exists for this "connection" (rules are specified during an initialization phase before the benchmark is started), the system creates an entry in the hash table for this connection to accelerate future handling of packets for this connection.

If the packet is determined to correspond to a NAT entry, the source address of the packet is altered as stipulated by the pertinent rule. The IP header checksum is then fixed to reflect this modification. Additionally, if the packet is a TCP packet, the TCP checksum is also updated to reflect the modification in source address. The translated packet is then sent onward. Packet Checking

The IP Packet Check benchmark performs a subset (essentially the IP header validation) of the network layer forwarding function of the Internet protocol suite as specified in RFC1812. The benchmark provides an indication of the potential performance of a microprocessor in an IP router system.

A TCP/IP router normally examines the IP protocol header as part of the switching process. The router generally removes the link layer header from a received message, modifies the IP header, and replaces the link layer header for retransmission. In this benchmark, the link layer header has already been removed and will not be replaced. Thus, all processing is done at Layer 3, on the assumption that lower level functions are handled by hardware or an interrupt service routine.

The benchmark simulates a router with four network interfaces. It initializes a buffer of programmable size (512 KB, 1 MB, 2 MB and 4 MB for reporting purposes) with IP datagrams. The header is always the minimum 20 bytes and is made up random characters except in the byte positions to be checked (IP version, checksum, and length).

Attaching Marks Once the benchmarks are complete, EEMBC will attach a TCPmark and IPmark to processors. According to Levy, the TCPmark shows the geometric mean of three TCP performance checks while the IPmark is the geometric mean of four IP packet checks.

Two processors have currently been certified using the benchmarking techniques. Freescale's 1.4 GHz MPC7447A processor delivered a 819.8 TCPmark score and 245.1 IPmark score. IBM's 1 GHz 750 GX processor, on the other hand, deliveres a 467.1 TCPmark and 286.1 IPmark. Additional information on the benchmarks can be found at www.eembc.com.