Cisco Nexus 6000: First Impressions

While attending Cisco Live in the London, I went to a session exploring the architectural details of the forthcoming Nexus 6000 data center switch. The Cisco Nexus 6000 closes Cisco's product gap in the cloud networking market and addresses the need for a leaf/spine networking architecture built around high-speed, low-latency Ethernet networking.

Cisco positioned the Nexus as a mid-range switch for aggregation, but I can't see who would buy it for that reason. That said, here are details.

The 6000 series includes line-rate forwarding for all ports at any packet size. Cisco claims 1 microsecond port-to-port latency via cut-through switching for both 10 GbE and 40 GbE. The series offers 31 concurrent SPAN ports (which I find awesome because it provides a much needed boost to network visibility and analysis).

It has a 256K entry host table shared for MAC, ARP, IPv6 ND, (*,G) multicast and /32 host route (which is good but not great, as Brocade does 384K/384K in its VDX chassis). The 6000 series also includes Equal Cost Multipath support for up to 1,000 paths.

The Nexus 6000 has a surprising amount of multicast support, including PIM Bi-Directional. It seems likely that this was intended for VXLAN support for overlay networking. For more on the relationship between VXLAN and multicast, see my post VMware's Dilemma: VXLAN or Nicira?.

The 6000 series comes in two models, the 6001 and 6004. The Nexus 6001 offers 48 10-GbE ports, plus an optional four 40-GbE ports. The 6001 comes in a 1 RU form factor with redundant power.

The Nexus 6004 is a pitched as an aggregation-layer switch in 4RU form factor with redundant power. It has 48 40-GbE ports in a fixed form factor. It can support a total of 96 40-GbE ports via four additional line cards. Cisco says future upgrades will include 100 GbE via line cards.

If you are new to 40 GbE, you may want to know that the QSFP interface is used for 40 GbE transceivers. The 40 Gb standard uses 4 x 10 GbE "lanes" that are multiplexed at the SERDES physical layer. Therefore, you can obtain a QSFP transceiver that acts a 4 x 10 GbE port and connect to existing Nexus 22xx series ToR switches.

40 GbE also uses a new type of connector, with eight fiber optic cores per interface. This will change how you cable your data center in the future.

Use Cases

The Nexus 6000 fits between the Nexus 7000, which is oriented toward service providers, and the 5500/2200 switches.

Today, the Nexus 7000 is the "kitchen sink" services platform. The NX-OS software on the Nexus 7000 provides MPLS, LISP, vPC/FEX, and other fancy but often buggy software features. It's expensive to run in terms of power consumption and maintenance, and very hard to upgrade and maintain due to its complex internal architecture.

Service providers and large-scale cloud providers have found that the Nexus 7000 isn't well suited for their needs and have turned to other providers. Cisco hopes to address this with the Nexus 6000. It will appeal to providers and large cloud implementations for fast, simple Ethernet switching. The Nexus 7000 will continue to be attractive to enterprises that perceive safety in buckets of features).

Cisco claims that the Nexus 6000 is the fixed format unit for 10G/40G/100G and returns Cisco's focus to core networking features. The Nexus 6000 has smaller rack space, low power, low latency and jitter, and a limited feature set.

While the maximum throughput of the Nexus 6000 is a hefty 7.68 Tbps, the Nexus 7000 could, one day, reach 36 Tbps. Most companies can't afford the 40 G line cards on the Nexus 7000 because of the fancy silicon that offers fancy features (and lead to high pricing).

Annoyances

While the Nexus 6000 series has some compelling features, I did find a few nits to pick. First is the airflow, which is back to front. Cisco says reverse airflow will come in a future release. Second, 100 GbE isn't expected to ship until mid-2014, which is a long delay.

Finally, its IPv6 scalability is limited. The TCAM must be using older designs for 32-bit addressing of IPv4, because IPv6 consumes multiple TCAM slots and reduces scalability to just 50,000 hosts. It's 2013 and IPv6 is still a second-rate technology at Cisco when it comes to hardware support.

First Impression

The Nexus 6000 series is aimed at customers that need high-speed Ethernet with a limited set of network services. For these customers it's a much better alternative than the 7000, which is too complex and unreliable for core Ethernet (though desirable for its comprehensive services and extended features such as QoS, MPLS, VDC and OTV). I'd relegate the Nexus 7000 to the WAN or campus edge and put the 6000 in the core for my networks.

What I see is Cisco slowly adapting to the new era of networking. While the Nexus 6000 makes the extended life support of the decade-old Catalyst 6500 even more confusing, equally, the Nexus 6000 will be welcomed by a significant segment of the market that is building public and hyper-scale cloud-based networks and needs speed, simplicity and scale instead of massive feature sets and fancy functions.