Cisco Overlay Network Bridges Distributed Data Centers

Cisco continues its march to complete its Data Center 3.0 vision with a proprietary method to interconnect remote data centers called Overlay Transport Virtualization (OTV), extending a layer 2 Ethernet network over a WAN. The claim is that OTV is simpler. Cisco also counted 10Gb Base-T Ethernet modules for the Catalyst 6500 and 4900 switches, and new IO modules for the Nexus 7000. Additionally, they are furthering their Wide Area Application Services (WAAS) product with VMware as well as automatic recognition, acceleration and optimization of SaaS services.

Organizations with multiple data centers typically face expensive and time-consuming work when they want to bring new services on-line or change existing services. The time to get a new circuit up and running, whether it's Carrier Ethernet or MPLS, can be weeks or months. Many WAN service plan changes incur additional costs, and that's just to get the network in place. Then, services have to be enabled. Cisco OTV is an overlay network that the company claims can be provisioned in minutes. Since OTV is an overlay on top of IP, the traffic can be routed transparently over any IP-based WAN connection. OTV is built to have low-management overhead, replication, is fault-tolerant and includes multi-path, optimized paths between nodes. OTV is available on Nexus 7000 in April, 2010 through a software upgrade.
Short on details, OTV uses a proprietary protocol to encapsulate Ethernet frames in IP at the network edge and transports them to the remote data center where the Ethernet frame is decapsulated. OTV extends a layer 2 network over the WAN. Broad control is built into OTV so that unnecessary broadcast traffic, including broadcast storms, doesn't traverse the WAN while allowing applications that require layer 2 connectivity, such as VMware's VMotion, to work.  The protocol allows for each Nexus 7000 to exchange control planes and divide MAC address knowledge among each peer automatically.

The downside of overlays is the additional protocol encapsulation overlay cost, in this case, typically 40 bytes per packet, which can add up fast. Also some applications, like storage protocols, may not behave well on an overlay network, where WAN characteristics, like delay and jitter, can vary. For example, carrier Ethernet WAN deployments that are used for high-capacity WAN connections tend to have more granular specifications for delay and jitter, while MPLS and IP WANs can vary widely.

In addition, since OTV is not encrypted, it is unlikely you will want to use this over an untrusted WAN. OTV packets may also pose a problem for firewalls and other network equipment that processes IP packets. Early implementations of IP VPN traffic, such as IPsec and PPTP, both of which encapsulate IP traffic, had issues with firewalls that couldn't process the packets properly. Whether OTV will suffer more limitations will be borne out in deployments.Cisco also enhanced WAAS to automatically detect and optimize SaaS applications, like Microsoft's SaaS applications and Cisco's own WebEx, to reduce WAN bandwidth and accelerate application delivery to end-users. IDC, cited by Cisco, states that 60 percent of cloud applications, including SaaS, are delivered to branch offices via central data centers over SSL. The WAAS enhancement decrypts the SSL-encrypted SaaS traffic, optimizes it and then re-encrypts and delivers it to end users in the branch. The WAN reduction can be 4:1 or more, and delivery from a local cache can speed up applications.

Cisco has tightened the integration of its Application Control Engine (ACE) with VMware's vCenter, providing a single pane of glass for load balancing and application acceleration. However, Cisco has yet to implement any automation integration between ACE and vCenter, such as managing virtual machines based on demand. Also, there is no ability to have application acceleration profiles move with a VMotioned virtual machine. Both are enhancements that Cisco is considering for the future, but they say they haven't seen significant demand for a ACE as a virtual machine.

Finally, Cisco added a 16-port, 10Gb Base-T module for the Catalyst 6500, which lists for $22,500, and an eight-port 10Gb Base-T module for the Catalyst 4900 top-of-rack switch, which lists for $3,500. Both modules will be available by the second quarter of 2010. Both let you use existing Cat 6a and 7 twisted-pair cabling for 10GB up to 100 meters, although at or near 100 meters, your cables and the connections have to be nearly perfect. Existing Cat 5e can be used for shorter runs up to 45 meters. For shops that need massive IP routing capabilities, two new modules, the eight-port 10Gb XL I/O module and 48-port GbE XL I/O module for the Nexus 7000 offers pay-as-you-go pricing to increase capacity and can support up to 1 million IPv4 routes and 256 thousand Access Control Lists (ACLs).