Have you noticed that WAN design is changing? Well, I think it is, and I’d like to share my thoughts.
A little history
Old TDM networks used to be “random” meshes of T1s, etc. Part of what drove the design was mileage-centric costs, so one would interconnect nearby sites and end up with a network that looked sort of like chicken wire fencing. Capacity planning was painful, as was multi-hops and latency. And the mesh-iness tended to aggregate traffic, thereby aggravating congestion and cost.
As Frame Relay then ATM came in, things gradually evolved to where they are today. We now tend to ignore distance, and just interconnect sites with MPLS or Metro Ethernet WAN “clouds.”
One of the old rules of thumb was to home anything circuit-like to the data centers; they generally don’t move or get replaced very often.
With MPLS, that became less of a concern. However, most applications moved to the data center (with the possible exception of portions of the federal government), so traffic flows were mostly data center-centric, excepting VoIP and other anomalies. That meant the WAN was also still data center-centric.
What's changed?
Two things:
Concerning that latter item, up until recently (#NFD13), I’d have said there were two current design approaches, but now I think there are three:
The trend is away from the centralized approach, except for geographically localized organizations.
Understanding the implications
Increased business use of the internet and SaaS, etc., leads many to conclude they want each site to access the internet directly, the decentralized model. The problem one then runs into is the cost of putting a firewall and other security tools at each site, plus managing all that stuff, let alone backhauling log information to a central location.
That’s great if you’re building an IT security empire; not so good if your budget is constant or shrinking. The costs go down if you’re willing to use your router as a firewall and security tool. I rarely see that. More often I see firewalls, etc. being used as border routers, which has its drawbacks (e.g., scaling IPsec VPN securely to many sites).
I’ll ignore DNS interception, whitelisting, and routing as a hybrid approach that just muddles the picture a bit. On another front, services like Zscaler and Cisco’s Umbrella attempt to address the distributed hardware cost/management issues for you by providing cloud-based protections. That sort of works but can add latency.
The problem with centralized internet access is that you back-haul all the internet traffic over your WAN, then provide egress to the internet. The consequence is bigger WAN links (internet plus data center traffic), and bigger central internet link.
Centralized internet does simplify some aspects of network security: fewer firewalls and security devices, fewer connection points to monitor.
The following diagram illustrates the bandwidth savings with decentralized internet:
The other consideration with the centralized approach is latency. Instead of taking the direct path from an end-user site to an internet business tool, traffic has to go to the central site, then to the tool. If the tool website uses a CDN (distributed content network), you’re pulling all those static graphics, etc. through your central site, rather than from somewhere near the end user. Slow!
The regionalized internet approach tries to strike a balance between the two. There are two variations:
The first of these is something Equinix and others excel at. One virtue of a good colo is inexpensive fast connections to various internet and cloud providers.
The second of these is the virtual equivalent of that. Challenges for the virtual version include limitations on the performance of a virtual router or security device, possible difficulty of getting inter-cloud provider connections, and cloud provider fees for egress traffic.
Here is what a regional approach might look like:
Data centers would get connected to the regional hubs as well.
The regional hubs would get interconnected by higher speed links. Good colocation sites generally have inexpensive connectivity to many high-speed providers.
With regionalization, you do have some of the same extra WAN-to-internet factor as in the above centralized internet design. The overhead is just split between the regional locations. Latency should be lower, in general, than with centralized — assuming a wide geographic spread.
MPLS is a challenge for this: The routing is unaware of distances and latency. Unfortunately, your CE router cannot specify the cloud egress PE for traffic engineering to regionalize traffic. DMVPN or mGRE is one way to tackle that: You can do regionalized hub-and-spoke topologies on top of MPLS. This is what some SD-WAN vendors are promoting. IWAN can certainly do that, too.
If you have high-speed links between the colo sites, that provides some geographic modularity and hierarchy in your network.
Another approach would be separate regionalized MPLS VPNs. (Other alternatives are left as an exercise for the reader.)
If you are trying to regionalize MPLS, you need to be somewhat aware of some of your carrier’s topology. If you’re trying to get traffic from Miami to Tampa, you may not want it detouring through, say, Atlanta to get there. The caveat here is that we may be seeing new WAN design approaches, but they may not play well with older designs or technologies. It’s on you, the WAN designer, to take all that into account!
If you use Internet-based SD-WAN or IWAN, you control the tunneling. Regional hub and spoke is then a distinct possibility.
Right now, SD-WAN to some extent has an incentive to like the regionalized approach, and vendor slideware suggests that approach. See for instance the #NFD13 video sessions.
Here’s why: Regionalized colo-based SD-WAN allows you to supplement the SD-WAN network with firewall and security appliances in the regional colo. As SD-WAN vendors add NFV capability to run virtualized security and other services on the same platforms (appliance or virtual appliances), more decentralization may become preferred.
In the near term, performance will likely define a sharp line between where virtual appliances are suitable, and where physical devices are needed. As general purpose computers’ networking performance improves, the virtual versus physical router boundary seems likely to move upwards, meaning physical routers could become a shrinking market. Having woken the reader up by scaring them, let me hasten to add that traffic growth might offset that effect a good bit. In other words, my crystal ball is hazy on this topic.
NEXT: Metro Ethernet and design notes
Bob Friday, Chief AI Officer for Juniper Networks, explains how the advanced technology is transforming operations.
Contact center leaders from 8x8, Awaken Intelligence, and 360insight discuss the importance of agent experience.
AI may force enterprises to rewire parts of their data centers so they are fully optimized to run such workloads. The question is do you use Ethernet or InfiniBand?
