A VoIP Wake-Up Call

Question Everything

When we began our VoIP pilot two years ago the technology was being hyped, with Alcatel, Cisco and other big-name vendors selling enterprise-size gear and visions of converged voice and data network nirvana. We ran the pilot to determine whether we should replace Syracuse's campus Centrex service with VoIP and what it would take.

Here's advice we wish we'd had up front: Study your voice and data network infrastructure before you start talking to VoIP vendors. We learned the hard way--but luckily early in the pilot--that our Syracuse network needed some key upgrades to support VoIP traffic.

Things started to get complicated when we moved pilot users from the TDM system to the IP network. Because voice can't tolerate normal traffic delays, we had to configure our switches and routers to handle the more sensitive IP voice. We also had to upgrade our LAN from half-duplex 10-Mbps Ethernet to full-duplex 100-Mbps with QoS (Quality of Service) on new Category 5E wiring. The old circuits were prone to voice no-no's--echos, dropped calls and other performance inconsistencies. The VoIP traffic needed new business rules, too, so we created rules that did tasks such as funneling IP phone problems in the pilot area to our own network engineers instead of to Verizon, our Centrex local telco.

We selected Cisco's AVVID VoIP architecture for the pilot mainly because Cisco had the most mature VoIP products and support at the time, and because Syracuse's backbone is built on Cisco switches and routers. It took us one year to build an IP voice design for our test (see Step by Step), with occasional help from Cisco.In your initial VoIP analysis, consider whether your network can handle voice traffic. The rule of thumb is you need at least a QoS-capable 100-Mbps Ethernet backbone but ideally you'd have Gigabit Ethernet. If your infrastructure is built on professionally installed and certified Cat 5E or higher wiring, your network's physical layer meets the minimum VoIP requirements. If you're thinking about adding gigabit to your legacy Cat 5 wiring, you'll need to retest it to ensure it can support the upgrade.

Other things to consider: Does your data wiring exist everywhere you'd like to add a VoIP set? What vintage are your switches and routers? Does your network have the benefit of a reliable, balanced and backup-protected electrical power-distribution system that can sustain uptime during an outage? Do you have network management tools, such as NetIQ Corp.'s VoIP Manager or Brix Networks' VoIP Test Suite, that can analyze packet-level performance and detect jitter, delay and latency?

A successful VoIP operation comes with a well-planned, modern and soundly implemented network. If your network is running on hubs or geriatric 10/100 switches and Cat 3 cables, replace them before you start setting up the VoIP call-processing hardware. In our case, we ditched our Cisco Catalyst 1900 switches in favor of new Catalyst 3500s to get the QoS, VLAN and power-over-Ethernet features we needed. If you have wireless networking, check whether your wireless access points support QoS parameters so the voice traffic gets priority when it hits the network's wireless segments. This may be tricky: Few of today's wireless devices come with QoS options.

Your network may need more than new switches and data wiring. Before you buy your VoIP equipment, build redundancy, failover and high-availability power distribution into the network. Uptime is crucial for voice: If you lose a segment of the data network, at the least it's inconvenient and at the worst you lose revenue and customers. But if that same segment carries your voice service and fails, you could face liability and would certainly lose clients. How you configure the Spanning Tree protocol and self-healing on the network and in the server farm can make or break the voice-call experience--and your VoIP infrastructure.

And don't forget your staff. If your helpdesk is divided into separate voice and data support staffs, be prepared for the sensitive process of merging them.

One of the most worrisome issues we grappled with during our test implementation was how to mesh our new VoIP solution with some big-dollar applications--account provisioning, cable-plant management, departmental telephone billing and helpdesk systems--that we had rolled out prior to the IP voice pilot. Even now, we still haven't quite figured out how to integrate those systems with a large-scale VoIP rollout. Our call accounting system, for instance, is based on our legacy Nortel Networks DMS-100, so it obviously wasn't a natural fit with Cisco's Call Manager-based AVVID.If your remote sites use different support applications from those your main location uses for voice-call accounting or helpdesk functions, be careful when you install your VoIP system in all sites. Different call-accounting and helpdesk functions can cause Excedrin headaches. Take inventory of your support software suites (and their dependencies and operational rules) so you can ask vendors and integrators what percentage of your existing network operations will work with VoIP and how much will need to be overhauled.

Depending on the size of your enterprise and the number of telephone extensions, you'll probably need to renumber some or all of your voice extensions. At Syracuse, going VoIP campuswide would require renumbering thousands of telephone lines. Each VoIP extension in Cisco's AVVID is a network device, so each IP telephone requires an IP address, too. That would entail adding thousands of new devices onto the network. And because of some limitations to our existing DHCP setup, accommodating these additional IP nodes would require a complete overhaul of our DHCP infrastructure.

Expect to add more servers with VoIP, too. As the Syracuse pilot progressed, we found that VoIP services, such as XML and call-center applications, typically require their own servers. We installed a separate server for the flashy XML applications we used for viewing the weather forecast and corporate directory on display screens on the VoIP handsets. We considered other high-end VoIP services, such as an IP call center, but even for our relatively small group of IP call-center users, it would have cost about $100,000, including at least four new servers.

With Cisco's AVVID Call Manager application, meanwhile, users enter login IDs rather than access codes for long-distance calls, a model that doesn't match our billing system. We identified a third-party call-accounting application that was a better fit, but it requires a separate server. So, in the pilot, we restricted the users to "trusted" IT staff members to avoid the call-accounting issues at the university. Their calls weren't tracked.

With all the additional Microsoft Windows 2000 and SQL 7 servers running VoIP applications, you'll have to deal with their administration, patches, licenses, upgrades, maintenance contracts and device security. At Syracuse, we always had four to eight servers running. Because VoIP blurs the lines between network and application, we weren't sure how to divvy up the work between the network and the server folks.Then there are the hidden costs of VoIP. They're all about options and licenses, and the expense can be substantial. Many standard PBX features, such as E911 (Emergency 911) service support, are optional with VoIP. Cisco's E911 server application, for instance, lists as a $5,999 option. This gets you 100 licenses, meaning 100 extensions can be part of the E911 service. Each additional license is $10, without volume discounts. It would have cost us more than $50,000 (in addition to the cost of the core VoIP components) to equip the entire Syracuse campus with E911 service.

And beware of additional hardware resource costs sneaking up on your budget. Case in point: Call Manager is responsible for call processing, database processing and management and other tasks, so there's only so much processor power to go around when it's time to place calls on hold or when multiple conference calls are under way. Call Manager's limited on-board DSP (Digital Signal Processing) resources handle conference calls and other advanced system functions. DSP performance is crucial at Syracuse because we have a large volume of outside calls: In the pilot, we found that when around 200 or more extensions were in use simultaneously with a lot of call conferencing and hold conditions taking place, the DSPs could get overtaxed. The only solution is to buy hardware. For a campuswide system, we would need multiple $19,000 add-in boards for our core switches to get more DSP power.

You may incur other unforeseen costs for your new VoIP environment that your vendor doesn't advertise, so hammer your sales engineer for details on the possibilities--especially if your network is growing.

History Lessons

If your IT group hasn't had to worry about QoS before, it will with VoIP. Voice traffic may not be bandwidth-heavy, but it is highly sensitive to latency, jitter and delays. Make sure you budget for VoIP-specific QoS training if these kinds of Layer 2 and Layer 3 QoS issues are new to you, as they were to us at Syracuse. That way, you can configure your network so voice gets top priority.Don't be shy about asking for outside help. Shelving the PBX and putting hundreds or thousands of calls across the network can be a daunting task, and few IT departments have the staff count and specialized talent to make the switch exclusively with in-house resources. Ask your VoIP vendor for integrator recommendations. Once you select an integrator, tell that person you want to know up front all the costs of implementing VoIP, and that your goal is to have minimal impact on users during the transition. At Syracuse, we handled our pilot project rollout in-house, but if we deploy VoIP across campus, we'll probably use an integrator to supplement our in-house skills.

It's worth looking into the history of outside contractors before you sign on the dotted line, too. VoIP is new enough that many consultants are still learning the ropes. The last thing you need is a green consultant using your deployment as a VoIP training ground so he can make a name for himself. Make sure you know your integrator's specific areas of expertise. There's a big difference between implementing an IP call-processing server and actually setting up a complex IP call-center application.

Although few midsize to large enterprises have made the jump to VoIP pilots or production systems, it's still the likely future of voice. Start studying, planning and upgrading your network now. That's the best way to prepare for deploying VoIP when the time is right.

Lee Badman is a network engineer at Syracuse University. Write to him at [email protected].How to Plan for VoIP

1. Analyze your network wiring infrastructure. You need Cat 5E wiring, minimum, for VoIP.2. Rethink your switches and routers. Aim for 100-Mbps Ethernet switched ports and backbone connectivity. VoIP is also a good excuse for putting in a Gigabit Ethernet backbone.

3. Check your electricity distribution system. With your network providing telephone service, you'll need stable and reliable power to your switches and phones.

4. Consider how you'll merge voice and data support staff into a unified group. Separate staffs would complicate things in a VoIP environment.

5. Make sure your network management tools can analyze down to the packet level so you can keep voice-call quality high.

6. Build a list of business rules and support applications that touch your voice service. Consider whether those rules would still apply if you implement VoIP, and if not, rewrite them to work with it.7. Choose an integrator based on his or her track record and ability to implement all the VoIP-related applications you want to deploy. Factor in your vendor's recommendation for an integrator, too.

8. Keep an open mind. It's going to be a learning process for both your users and your support staff.