Reality IT: Pre-Purchase Product Testing Can Save the Day

Not a Bug, a Feature

When I first started at ACME as the network manager, we were implementing an intelligent-routing application for our call center. The decision to buy had been made before I came aboard, and it wasn't a good one.

Our non-IT decision-makers had taken the word of the vendor (a very big name in telecom) that this product would run on our managers' workstations with no problem. Well, it would have--but only if the workstations weren't running the IPX/SPX protocol and we were a Novell NetWare shop. The application required NetBIOS to run properly, and it took us an extra month to jury-rig a solution that let all the protocols co-exist. The project was delayed by almost two months, but a few simple prepurchase tests could have exposed the problem before it affected our business.

Just Fell Off the Turnip Truck

I've heard some sales pitches that were so full of holes, you could use them to grate cheese. One vendor's "installed base" consisted of two companies. Another vendor's software required massive servers that would make NASA envious. A developer once proposed to create a custom application for us at a cost of more than a million dollars--even though we were evaluating off-the-shelf applications that fit our needs well. Still another vendor would not agree to use our test data--even though two of its competitors already had.But when the testing starts, the presentations stop, and good vendors will help you prove the viability of their products in your environment. Some vendors will give you a production version of an application to test, rather than some watered-down evaluation copy. A number of them will offer demonstrations that are customized to your environment. Occasionally, you can get several vendors to do a live "shoot-out" of their products, testing their performance against each other in a real-life setting. Not all vendors will agree to these approaches, but you'll never know unless you ask.

An Ounce of Prevention

A thorough product evaluation requires lots of research. At ACME, we use vendor questionnaires, analyst reports, magazine reviews, consultants, product demonstrations and shoot-outs among top contenders. We also maintain a lab that includes separate environments for development, testing, staging and production. In recent years, we've been using VMware (which provides a virtual infrastructure) to get more from our development/ test environments: It rocks as a way to set up a test bed with just a few boxes. If you're not ready for VMware, consider other easy options--for example, some of our IT people are doing simple product testing on spare computers in the shop.

We're all busy in IT, and it can take a lot of effort to test products in your environment. But the work pays off in the long run. Just ask anyone who's ever deployed a product without first giving it a trial run.

Hunter Metatek is an enterprise IT director with 15 years' experience in network engineering and management. The events chronicled in this column are based in fact--only the names are fiction. Write to the author at [email protected]. 0