Performance Monitors that Get Results

Calling All Collections

Performance-management products collect data passively, actively or in some combination of both. Each approach yields different results.

Passive data collection relies on actual traffic and derives performance statistics from real users and network and system conditions. An agent on an end user's PC, for instance, watches the network stack and reports throughput back to a central server. Lucent Technologies' Vital Suite uses passive agents. Passive probes monitor a data stream through a span port on a switch or a tap on a network segment. Probes from Compuware and WildPackets promiscuously decode the traffic to count on anything from packets to transactions.Active measurements insert a known load through the replay of a transaction, such as getting a Web page, making it possible to compare actual results with expected results. Active methods send actual packets--sometimes transactions and sometimes just dummy payloads--through the network. Examples of active methods are found in the Web site-monitoring services provided by Gomez and Keynote Systems, which have agents around the world periodically downloading Web pages, measuring and recording results.

You'd think the outcome would be similar in active and passive cases, but in the absence of users requesting pages, the passive approach isn't aware of performance degradations. With active agents, in contrast, you can run a set of transactions across a newly upgraded application and be assured that everything is ready for real users.

Agents, More or Less

Agents are little busy-body programs that run on servers, clients, switches and routers collecting performance data. Agents might collect data about CPU utilization, application processes, database queries, network errors or even execution of transactions.In response to concern about having to maintain agents on devices, some performance products use a remote data-collection method referred to as "agentless." The agent is still there, but typically it's an existing one that doesn't require any care beyond turning it on. The most common agents of this type are SNMP MIB II agents found on routers, switches and hosts.

SNMP agents suffer from bad press because they use open, readable passwords (at least in versions 1 and 2) and, like other agentless systems, the data they gather isn't specific. Yet some agentless approaches can provide many details about a systems environment. IBM's WebSphere performance interface, for example, monitors applications and processes running in the application server.

Passive probes don't give you any of the hassles of agent access, installation or maintenance--they just sit on the wire collecting data. And they usually come as appliances, so all that's necessary for implementation is network access and an IP address.

But probes have two inherent weak points. In a switched network with lots of segments, you need many probes or a large probe on an aggregated backbone segment. Because probes like NetScout Systems' nGenius can gather RMON Layer 2 metrics, aggregated segments provide local Layer 2 metrics and Layer 3 and above metrics for packets originating off the local segment. This is fine for most performance-management requirements, which focus on IT services at and above Layer 4, with the most valuable performance data coming from application transactions. However, aggregated segments have such high traffic volume that collecting so much data is like drinking from a fire hose--you have to manage what data gets filtered out. This care and feeding is an often overlooked part of probe-based data collection, and may require professional service and training to tune and maintain.

The question of which measurement approach best meets your needs is answered by determining a combination of factors: on which layer your application is operating, its physical architecture and who needs the data.You may have a full-blown, multitiered, centrally located e-commerce site, where client experience must be measured in relation to promotional and merchandizing efforts. It's complex, but it's all fed from a central switch, making it possible to set a passive probe on a span port to see all the application traffic.

Or your application may be required to deliver 256 Kbps of consistent bandwidth for streaming media. You'll benefit from agents on both sides of the link to test that the network is being held steady.

Data needs are usually determined on a parochial basis. If the network folks are buying a measurement tool, they're looking for monitors that search out network faults and clearly prove the network is running well. So, SNMP data collection as well as proprietary switch and router MIBs using agentless remote data collection will be the right choice. Application awareness is limited to knowing the application is running on the server, so a download of a Web page using a robotic active transaction will do.

At the other end of the spectrum, business owners want to measure what traffic is driving certain revenues--a feat that still is not really possible with performance software. What is possible for this group is monitoring key transactions in real time to measure the customer's experience. For this sort of deep application/transactional dive server, agents with specific transactional capability, or probes with the same deep knowledge of your transaction type, are in order. Both are available, but they require professional services and a top-down team effort.Performance-management offerings continue to evolve with more and more data collection available. The mountain of measurements provided will be useful to your organization. Just be sure you know what to measure before you start.

Bruce Boardman, executive editor of Network Computing, tests and writes about network and systems management. Write to him at [email protected].