WAN Services

AT&T, BT and Qwest Communications International agreed to take a shot at our RFI. Masergy and MCI declined, while BellSouth, Broadwing Communications, CSC, EDS, Equant, Global Crossing, Level 3 Communications and Netifice Communications didn't respond.

Why MPLS

Amr Ahmed, internetworking practice director at Greenwich Technology Partners who has spent years immersed in MPLS, helped us analyze this RFI. Ahmed points out that frame relay is a circuit paradigm and MPLS a services paradigm: If you use MPLS to re-create your frame relay connections, you probably won't save much and may even pay more, given the price wars swirling around frame relay. Rather, MPLS' value stems from the integration of services, like voice, video and Internet connectivity.Here's a quick refresher on MPLS technology to help frame (pun intended) the two technologies. MPLS CE (customer edge) routers connect to PE (provider edge) routers over static routes or eBGP (External Border Gateway Protocol) updates. Routes to remote locations are advertised within eBGP using standard interior routing protocols, such as OSPF or ISIS, as opposed to frame relay PVCs (permanent virtual circuits), which are configured for point-to-point or point-to-multipoint using static circuits.

The PE is configured with a VRF (VPN routing and forwarding) table that relates to the locations represented by the internal routing protocol. PE routers distribute these "VPNs" into their core and throughout their networks using Multiprotocol iBGP (Internal BGP). Essential to this technique is a separation of control between enterprise private network routing and provider networks to let enterprises and providers manage their respective networks without the level of alignment frame relay PVCs require. Enterprises can choose which sites are connected and how their traffic will be marked for which class of service. It's worth noting that CoS applies only to the customer's traffic, not against all traffic on shared portions of the provider's network. Providers can focus on the aggregation of many enterprises' traffic, without having to provision and manage to the service agreements for each separate circuit. Once customer networks are plugged in and static routes set up, MPLS provides a full mesh of connectivity that would take a lot of work--and expense--to achieve with frame, especially for a company with a lot of locations, like TacDoh. For more MPLS basics, see "MPLS VPNs: The Real Deal".

TacDoh is aggressively evaluating the elimination of frame's point-to-point and hub constructs based on distance and LATA (local access and transport area). All the service providers that participated in this RFI based their proposals on Layer 3 MPLS VPNs, which create meshed circuits across the provider network to allow flat-rate, fully meshed connections.

Security Flaws?

One drag on MPLS adoption is nagging concern over security. Does MPLS pose a greater risk than your existing WAN technologies? When addressing security in MPLS compared with frame relay, you must understand the different planes--control plane and data plane--on which traffic traverses the network. MPLS separates these planes, same as frame services, and as you'll see in our RFI responses, MPLS providers take extra steps to secure the management plane and protect against intrusions.

So why do security worries regarding MPLS still persist? We believe it's the use of the VPN nomenclature. IPsec and SSL VPNs encrypt traffic before routing it over the public Internet, and MPLS VPN connectivity is obtained through the creation of virtual segments and by virtualizing private networks over shared media--so MPLS should have the same privacy applied as IPsec tunnels, right? Wrong. Not because MPLS traffic doesn't flow on the Internet--which, technically, it could--but because public Internet traffic doesn't flow within an enterprise's private MPLS mesh or cloud.

Still, the service providers answering our RFI were sensitive to security concerns. Each assured TacDoh that its MPLS traffic would travel on a completely separate path from the riffraff Internet traffic traversing their networks. Sounds good in theory--but in reality, could your provider accidentally provision a private LSP (label switched path) onto the public Internet, or vice versa? Of course. If you don't trust your provider to avoid these kinds of errors--which are also possible with frame relay--then get a backhoe and a spool of fiber and start digging, or get MPLS and benefit from the rich set of management and self-help tools available to monitor provisioning.

Finally Decide

All the service providers responding to our RFI had different strengths. TacDoh's current WAN is a typical hub-and-spoke frame network, carefully laid out over regional service provider PoPs (points of presence) to leverage monthly recurring circuit charges while maximizing connectivity (see "RFI Scenario").AT&T has the most mature offering, evidenced by many integrated services and a wide reach using its AT&T global network, with its MPLS VPN architecture based on the RFC 2547bis standard. BT has the broadest coverage, with very well-developed DSL and Ethernet broadband access and, as expected, a significant MPLS footprint in Europe. However, its menu of MPLS services is skimpy. Qwest split the difference with excellent reliability, a good service offering and, without a doubt, the best price.

Although our Report Card shows AT&T and Qwest in a numerical tie, AT&T takes our Editor's Choice award. Its pricing was nearly as heart-stopping as TacDoh's deep-fried goodies, but AT&T led in MPLS services, the focus of this RFI. Qwest took home our Best Value award and should be on anyone's shortlist. If services are not at the top of your must-have list, check out our Interactive Report Card to tailor our scoring to your reality.

In 2003, AT&T declared MPLS to be its strategic direction, and the provider has been busily adding services, PoPs and interprovider peering ever since. In fact, AT&T's advanced services were by far the most impressive and included multicasting, VoIP (voice over IP), IP video bridging, broadband Internet access, traffic encryption, good remote-access options, and a business portal offering visibility into your MPLS network and self-service, all available natively through the MPLS cloud.

For TacDoh's voice needs, AT&T specified its Voice DNA set of services. Network integration looked good; the company says it supports existing legacy PBXs, IP PBXs and IP Centrex PBX services. Videoconferencing is also integrated into AT&T's MPLS service, and point-to-point/multipoint links and integration with external ISDN connections are supported in the cloud. All are configurable from the AT&T portal.

The RFI was short on specifics regarding broadband access, but AT&T touts 1,000 global PoPs. TacDoh liked that AT&T provides IP multicast services within the MPLS cloud--this removes the requirement that TacDoh's routers be configured to support multicast. Of course, TacDoh's internal switches would have to support multicast, so an enterprise administrator must still do some configuration.

In addition to the aforementioned performance and video services, AT&T's Business Direct portal provides service alerts, personalization and collaboration. Its AT&T VPNMon monitors current MPLS VPN status, showing impairments and edge-to-edge links through real-time display of PE-to-PE MPLS iBGP sessions.

Remote network access from broadband and dial-up connections are integrated into AT&T's MPLS offering. RADIUS, SecurID and SafeWord access protocols can be deployed to manage access.TacDoh would provide Internet access to its branches from within the MPLS cloud using AT&T's Unilink service, which provisions separate logical channels. Access can be tailored depending on geographical, organizational or business need. There is a monthly charge per logical channel, but not having to set up Internet connections at each local branch will take a load off TacDoh IT's buns.

AT&T specified standard point-to-point IPSec VPN support using Cisco Systems LAN-to-LAN connections. TacDoh really likes AT&T Virtual Interface Gateway, which lets external suppliers and customers connect securely via IPsec into AT&T's MPLS fabric, further extending MPLS connectivity.

Forget the CDR (Committed Data Rate) restrictions placed on frame relay bandwidth--MPLS is much more flexible. All the providers have CoS offerings; AT&T was second to BT for defined CoS supported from the CE into the PE.

AT&T has four CoS levels available, with all four or some subset usable as determined by need. CoS 1 is meant for real-time, delay-sensitive traffic, like voice and video. The other three classes have defined minimum bandwidth levels, and TacDoh can simultaneously transmit over all classes. Unused bandwidth is released for use by other classes.All the providers offer traffic classification based on source and destination IP, TCP/UDP port, input interface, application protocol and DSCP marking, and all support marked traffic from the CE device. These DiffServ Control Points really just move traffic from the CE into the provider's core with reliable bandwidth without necessarily specifying bandwidth reservations across that core. However, all the vendors described their CoS SLAs (service-level agreements) as having end-to-end reliability. These are important attributes, even though they aren't differentiators for TacDoh, which maintains control of its CE routers. Still, honoring of premarked traffic means TacDoh won't have to wait for the service provider to tweak its CoS configurations. And though none of these vendors indicated an additional charge for re-marking traffic within an existing CoS offering, TacDoh wants to avoid incremental configuration charges.

Within the United States, AT&T offers a 39-ms round-trip delay and a 0.1 percent one-way packet loss, and for CoS 1 a 1-ms jitter. It doesn't artificially restrict or charge for CDR. As with all the providers, the access circuit is the only limiting factor. And AT&T's use of differential queuing is similar to that of the others.

AT&T has an SLA restriction for the available bandwidth on any circuit. Delay increases nonlinearly across a circuit as utilization rises, and AT&T speaks to this by referring to "Managed Access Connection" bandwidth. We expect that about 70 percent of total bandwidth should be guaranteed.

AT&T seems rightfully proud of its BusinessDirect portal. Available 24/7, the portal enables self-serve configuration, including provisioning of new sites and moves/adds/deletes. It also offers reporting, including accounting and billing, site-to-site network statistics based on CoS and a network topology map. TacDoh could publish reports on four weeks' worth of hourly average performance data on a per-port basis. In-and-out utilization, errors and discards are collected in five-minute samples. CoS overlays are also displayed, showing the same data by CoS classification. Changes in the CoS profiles are customer-controlled through the portal and take effect within 24 hours of the request.

Like the other participating providers, AT&T supports any-to-any and partial meshes within its MPLS fabric on a single access circuit through PVC or frame encapsulation over private line. To enable partial meshing to create partitions or, in MPLS parlance, communities of interest, all the vendors create multiple VRF tables, on virtual subinterfaces of the router. So while it looks like a separate physical path to the traffic, MPLS just tags the traffic destined for each community of interest differently, thus partitioning the network. All the providers allow intercommunity connectivity--heck, if the CE is managed by the customer, they can hardly prevent it!

AT&T's proposal did not reveal exactly how many MPLS PoPs it has. Instead, its response touts 1,000-plus global PoPs, about 600 of them in the United States. AT&T says TacDoh is covered, but it's impossible to assess from its answer how much backhauling is required to support our MPLS needs. Financially, TacDoh doesn't really care where the PoPs are because MPLS connectivity is flat-priced compared with frame relay, but if the distance is significant--say, crossing state or country boundaries--backhauling by a provider to an available PoP may increase delay and reduce circuit reliability. AT&T has interconnectivity arrangements with other carriers, at Layers 1, 2 and 3. Again, there were few specifics in AT&T's response. Given TacDoh's strong Asian presence, we were happy to note a peering arrangement with two Chinese providers that adds 89 nodes.

AT&T's Network Disaster Recovery program was notable. It includes a fleet of globally positioned, manned recovery trailers that can bring a PoP from "smoking hole" to operational in 24 to 168 hours. Not as notable is the fact that AT&T has not yet made MPLS FRR (Fast Reroute) operational, as Qwest has. Instead, AT&T still relies on Sonet failover and Layer 3 route tuning for resiliency. It did indicate that FRR will be in trials by the end of the year.

When we asked about deployment time and cost specifics, AT&T started to tap-dance. It rehashed the features and added some details about the management of services, but in no way outlined how long or what it would take for TacDoh to convert to its services. This is one area where Qwest excelled. See AT&T's complete response.

AT&T VPN, a monthly recurring $2,132,821 without access-circuit costs. AT&T Corp., (800) 222-0400, (908) 221-2000. www.att.comQwest did its homework and sent us a meticulously crafted, detailed response that went a long way toward reassuring TacDoh and guiding its MPLS strategy. For

this reason alone, TacDoh placed Qwest on its shortlist, anticipating an exacting and clear working relationship. Besides being a couple of factors less expensive than AT&T and BT, Qwest's offering has a good mix of service and MPLS technologies running on its production network. For example, Qwest has beaten both AT&T and BT in implementing MPLS Traffic Engineering and FRR.

Qwest's MPLS service pie has a few more pieces that BT's, but not enough to beat out AT&T given TacDoh's focus on services. One bright spot is its integrated voice service; OneFlex Host VoIP is a fully hosted VoIP offering that doesn't require an on-site PBX. Unlike AT&T, Qwest doesn't tie together legacy and Centrex offerings, but its service would leverage TacDoh's MPLS connectivity. As with AT&T and BT, Internet connectivity is available from within the MPLS cloud, but IP video bridging, broadband access and multicast support are all still on their way.

Currently Qwest offers three QoS classifications, with a fourth planned for later this year. A high-priority queue using LLQ (Low Latency Queuing) is suggested for voice and video, while the other two queues handle the rest of TacDoh's traffic through Weighted Fair Queuing. Qwest supports a number of queue configurations designed to support high-priority packets without starving lower-priority traffic. Again, Qwest's detailed explanation helped TacDoh understand how to match its traffic to the correct CoS selection.

Qwest recommends doing reporting by means of the probes it offers with its managed CE service, which uses Visual Networks products. TacDoh's network isn't going to get fully instrumented; more likely, some subset of representative or critical end-to-end CoS monitoring will take place, similar to AT&T and BT.

Qwest offered an impressive 25-ms CoS delay, 0.5 percent packet loss and 2-ms jitter, numbers that are comparable to--and, in some cases, better than--those of the other vendors.

TacDoh is interested in the Qwest Control self-service portal for provisioning and performance monitoring, which is available as part of the MPLS service and uses Visual Network probes placed at CE locations to provide performance reporting for Qwest's managed service. Qwest Control supports viewing of bills, performance and current network status and provides CoS provisioning, a link to Qwest support and the ability to enter and track trouble tickets. This is comparable with AT&T, lacking only support for reporting CoS mapping and usage.

To connect multiple MPLS clouds, Qwest is banking on inter-BGP Autonomous System, part of the BGP protocol defining separate networks in the Internet. This, of course, requires private peering by the provider with other providers and is an indication of Qwest's MPLS maturity.

Qwest, like the other vendors, supports RFC 2547bis with redistribution of iBGP for routes. It made a point of telling TacDoh that private PE routers are not on the Internet, an assurance meant to quell the paranoia that traffic is somehow being mixed up on the Internet. Qwest, unlike AT&T, does not yet support Layer 2 MPLS point-to-point connections, though it says it plans to in the future.

Qwest uses RSVP-TE to engineer traffic flows, which is a good thing. Its response goes on to indicate that it overengineers its circuits, upgrading when the 95th percentile of performance is 40 percent or if peak utilization reaches 60 percent. This sounds impressive, but it's similar to AT&T's reference to a "Managed Access Connection" bandwidth.Qwest advertises 98 IP PoPs, but only 11 are MPLS-capable. This means some of TacDoh's circuits must be backhauled. Notable are its wholly owned PoPs in Hong Kong, Singapore and Tokyo.

One consistency throughout Qwest's response was a willingness to go into specifics. For deployment, the provider laid out what locations should be converted first and what costs were included. This seemingly obvious inclusion in an RFI reply was not part of AT&T's or BT's submissions. Qwest's global network strategy is to interoperate with other service providers, grow its network reach and add services. Examples are recent PoP adds in Asia and MPLS peering with InfoNet. Given the purchase of InfoNet by BT, Qwest says it plans to enter into a similar agreement with BT. Take a moment to look at Qwest's RFI response. We think it's a prime example of the species. Also, see Monthly Cost Geographical Cover.xls.

Qwest IQ Networking, a monthly recurring $235,638 without access-circuit costs. Qwest Communications International. www.qwest.com

BT's got network for sure, and Europe is clearly home court. But the provider's U.S. presence is also impressive: BT claims more than 47,000 MPLS ports cranking across 2,000 customers, with additional ones on the way. And it isn't afraid to show where all this access is located--BT declared the location of its U.S. PoPs, both legacy and MPLS, in stark contrast to AT&T.

However, BT's advanced services were just OK, falling short of the rich options AT&T provided. Broadband access was the only standout--BT offers good coverage, especially in Europe. It's also the only vendor to have a widely deployed Ethernet option in addition to DSL. Traffic encryption and Internet access services are available and useful, but they don't distinguish BT's MPLS service offering from AT&T's and Qwest's. What does set BT apart is the lack of voice, IP video bridging and multicast support. All are on BT's road map, but for now, the deficiencies were too obvious for TacDoh to ignore, especially voice: BT cannot currently provide TacDoh with voice and/or call-processing services, though it does have a European voice presence that it says is expanding to other countries.

On the plus side, BT has the most classes of service, with six DiffServ Code Points mapped through its network. The control, queuing and mapping of CoS from the CE to the PE were similar to AT&T's. And BT adds protection for high-priority traffic while queuing lower-priority discardable packets. CoS performance reporting is broken out by class, showing averages and peaks. The core network is measured against CoS SLA for jitter and packet discard. BT supports end-to-end monitoring, just like AT&T does, and we were impressed that 30 sites could participate in end-to-end tests of CoS. BT indicated it does offer service levels for specific classes of service, but it declined to specify numbers.

Unlike Qwest, BT has yet to implement native MPLS traffic engineering protocols, such as RSVP-TE (AT&T says it's adding this by year's end). TacDoh sees RSVP-TE as a core part of assuring traffic control across the provider network.

BT clearly spelled out TacDoh's deployment costs, which seemed reasonable. What's not so reasonable is the time for a change to the network--a rather long five days. This is especially untenable when compared with AT&T's and Qwest's self-service portals, which offer next-day turnaround.

In addition, BT's global network strategy response had a glaring deficit in TacDoh's eyes: the lack of any interprovider label exchange. Despite the fact that Qwest is working on an agreement with BT, BT is not mentioning any interprovider connections, preferring to route traffic on its own network as much as possible. And even though Qwest is partnering with BT, BT is not admitting to returning the favor. This means BT would be the be-all and end-all for TacDoh traffic--some vendors do prefer to service all customer traffic, sort of a one-stop shop.

See BT's complete RFI response.

BT MPLS, a monthly recurring $1,171,415 without access circuit costs. BT Group, +44 207 356 5000 (United Kingdom), (800) 331-4568, (646) 487-7400 (United States). www.btplc.com

Bruce Boardman, executive editor of Network Computing, tests and writes about network and systems management. He has 12 years' experience managing networks and distributed computing for a financial service provider. Write to him at [email protected].

Amr Ahmed is internetworking practice director at Greenwich Technology Partners, a vendor-independent IT professional services firm. He provides strategic architecture consulting for global enterprises and and specializes in network-based MPLS VPNs, traffic engineering, videoconferencing and data-center high-availability networking. Write to him at [email protected].

R E V I E W

WAN Services

Sorry,
your browser
is not Java
enabled

Welcome to NETWORK COMPUTING's Interactive Report Card, v2. To launch it, click on the Interactive Report Card ® icon above. The program components take a few moments to load.

Once launched, enter your own product feature weights and click the Recalc button. The Interactive Report Card ® will re-sort (and re-grade!) the products based on the new category weights you entered.

Click here for more information about our Interactive Report Card ®.

TacDoh Corp. is a 7-year-old food services company that provides the best in deep-fat fried snacks around the clock through an expanding network of retail walk-in and drive-up outlets. This waist-widening endeavor is supported by 128 management sites, all linked with frame relay. Of these, 114 are in the United States. The top of this hierarchal network has three data centers connected to five regional headquarters. Below and connected to the headquarters are 20 district hubs. At the lowest level connected to the district hubs are 100 territory sales offices.TacDoh aims to support its applications efficiently and cost-effectively by replacing its frame relay network with MPLS where possible. The key decision point will be available MPLS-enabled services, including IP multicast support, VoIP (voice over IP) call processing, IP video bridging, Internet access, broadband connectivity for small locations and remote users into the MPLS cloud, and traffic encryption. Other considerations include the providers' MPLS backbone architecture and topology, total and MPLS PoPs (points of presence), contingency and backup systems, CoS (class of service) and traffic-classification offerings, deployment time, future plans, and of course, price.

(vital stats)

In its second appearance, our fictional fried-food seller TacDoh Corp. is looking to slim down its WAN costs by moving from frame relay to MPLS (Multiprotocol Label Switching). The company realizes there may not be an immediate reduction in price, but believes MPLS will pay off over the long term in richer real-time voice and video communications and lower network-management costs as a result of the advanced services offered by top MPLS service providers--perks like easy provisioning of Internet access and videoconferencing links, support for legacy Centrex and VoIP (voice over IP), and easy self-serve provisioning and billing through the provider's portal.

AT&T, BT and Qwest agreed to take a shot at the RFI. BT boasts an impressive reach, with 1,100 PoPs (points of presence) in 80-plus countries. However, its advanced services lag behind those of AT&T. Qwest does better on services and is a price leader, earning it our Best Value award, and we liked its RFI response the most. In fact, Quest and AT&T tied for first in our Report Card. However, we award only one Editor's Choice, and this time AT&T overcomes a high cost to take it: TacDoh's focus on MPLS services meshes well with AT&T's offering, which is mature, expansive and packed with a rich set of MPLS service options.