Strategy Session: Judging Products

Most of you have been reading Network Computing for a long time. The way we figure it, reader loyalty is a strong indication that we're serving your needs. But as we change and upgrade the way we deliver information to you, we've spent a few nervous moments wondering if we might alienate readers who expect things to stay the same.

Most recently, we've decided to do away with the letter grades that have been the signature of Network Computing reviews for nearly a decade. You've seen fewer of them in recent months, and now you won't see any. But don't take that to mean we've given up on detailed product analysis--in fact, it's just the opposite. We want to do a better, more nuanced job of describing the strengths and weaknesses of the products we test than simple letter grades allow.

Letter grades served Network Computing well when we were judging the packet-forwarding performance or SNMP conformance of basic networking gear. When the point of the product is affordable performance and simple compatibility, why not assign a letter grade and be done with it? Thing is, we're very unlikely to judge products simply by those features again. Instead, we concentrate our energies on products and technologies where choices are less obvious and where product details are anything but clear.

Product complexity is one reason to avoid letter grades, but it isn't the only one. Another is the maturity of the industry. You may prefer Lotus Notes over Exchange and so might we, but products have evolved the way they have for a reason, and attempting to letter-grade them doesn't serve anyone's interests. What you need to know is detail so you can choose the right products for your environment. Which product has a better e-mail reader, and why? Which is a better application platform, and why? These are the sort of detailed evaluation points that might actually help you choose a product.We've also learned that what makes a product right for one industry--say, financial services--could make it exactly wrong for another industry. Large, multinational enterprises have different criteria than do midsize manufacturers or retailers. We believe that by exposing the detailed process by which we evaluate products, you can more effectively assess products for your own shortlist. And as always, we know that when we find a product lacking, our reviews help provide you with the ammunition to force vendors to address those shortcomings. We hope you'll like our new evaluation style, and we're willing to bet you won't miss letter grades--not even a little.

Art Wittmann is editor in chief of Network Computing. Write to him at [email protected].