Affordable IT: Network Monitoring update from September 2004

Arranging to be notified when systems fail is often the first piece of network management that organizations tackle--and for good reason. Whether you're big or small, monitoring when your applications, processes, and network switches and routers are going south is basic network management.

Still, it can be difficult to know where to start with network management. The term gets a bad rap for being complex and expensive. To make matters worse, it's inconsistently applied. What it really means is network, systems and application management rolled into one.

Management is a loaded term in itself. The FCAPS (Fault, Configuration, Accounting, Performance and Security) model is often used to organize and quantify what is meant by "network management." It, along with the ITIL (IT Infrastructure Library), a best-practices definition and part of the ISO 9000 certification, defines the pieces and processes of managing all things IT.

For this brief tour, however, we're going to cut to the basics: how to find out right away when your stuff stops working.

The process of checking, logging and notifying is generally accomplished using Ping and SNMP. There are proprietary products and clever ways to determine whether a service is up and running, like running telnet into a device to see if it's still there, but the basics shared by almost every network- management tool are these widely adopted and available methods.If you're handy with scripting, you can run a Ping or get your own SNMP data without any additional network-management tools, free or otherwise. Ping executables are included in every operating system, including Amiga! SNMP is nearly as ubiquitous.

SNMP offers a huge stockpile of information about everything except what you had for lunch. We won't go into the nitty-gritty details of SNMP and its data structure, called MIB, but you'll find traffic to errors, CPU cycles to OS versions and just maybe the LDL in that BLT (read an SNMP tutorial).

If you've heard that SNMP is a security hole and are concerned about turning it on, you have a couple of choices. To play it safe, you can leave SNMP off, using Ping and maybe SSH to link to switches and servers. But we prefer Option 2: turning SNMP on responsibly.

While it's true that SNMPv1, which represents 99.99 percent of implementations, sends the Community String (what passes for security) in clear text over the network, telnet and FTP do the same thing. Still, this is no excuse--so use methods like SSH or secure FTP whenever possible.Even though SNMPv3 addresses the clear text community's shortcoming, making it a secure network-management protocol, it isn't a reality most networks can currently deal with. SNMPv3 involves some heavy lifting to make the security manageable and will take time and product (read: dollars) to do right (read a tutorial on SNMPv3).

Keeping It Simple

More practically, you can take some simple steps to protect your network while using SNMPv1.

First, get and keep systems patched. Then change the default Community Strings.

When SNMP v1 was first breached, the press fanned the flames of fear but didn't mention that most of the systems hit didn't have current patches that addressed the SNMP problems and that they were using vendor-default community strings. SNMP documentation often refers to the read community string as Public and the write string as Private. And this is exactly what many vendors' default community strings are named. Adding to the risk, many newbie administrators read documentation and think they must have Public/Private as the actual string names. Thus, it's a safe bet for an attacker that a network scan will show up some Public/Private accessible devices. This, of course, is human error, not a failure of SNMP.An even safer, albeit more complex, strategy includes creating a management plane that carries only SNMP and management traffic. By separating the management network from the production data network, which could be accomplished using VLANs or separate NICs, access can be limited to particular networks.

SNMP is an ideal monitoring approach for budget-constrained shops because of its wide availability: Everything from Windows and Web servers to switches and source code has baked-in SNMP support. In addition, lots of good tools are readily available for free or at a reasonable price. For example, the Windows resource kit includes the command-line SNMPutil and SNMPmon utilities. These will let you get SNMP variables from network devices and other servers, and SNMPmon can actually log results over a period of time.

On Linux and Unix, a search for SNMP utilities will return a landslide of offerings. SourceForge.net and SimpleWeb.org are both good places to look for SNMP tools specifically and network management in general. Linux dominates at both sites, but there's no shortage of Unix and Windows network-management applications.

One of the first tools you'll run across is MRTG (Multi Router Traffic Grapher). MRTG may not be the most full-featured free network-monitoring software out there, but it's widely deployed, and its active support base makes it a good place for NSM newbies to get started. It runs on everything, even Windows 98, and most beginners can start monitoring network devices via SNMP within an hour or so (download MRTG).

Another worthy piece of free network-monitoring software is Big Brother, now owned by Quest Software. Like MRTG, it's widely deployed, with an active user community supporting it. Quest, while creating a commercial version with dedicated support, is also maintaining the free version of Big Brother and its support network.

If you're more comfortable with a commercial offering, Ipswitch's WhatsUp Gold, SolarWinds.Net's Network Management Toolset and Neon Software's CyberGauge are all easy on the pocketbook while packing some serious features (see our review of inexpensive commercial offerings;).

Pick Your Battles

When deploying tactical network monitoring in a small environment, consider the following.

For starters, don't waste your time on complex automated discovery, or autodiscovery. If you have five or 10 routers or a few hundred nodes, you're better off doing a manual inventory of your network assets. Do take a look at how often your network changes, of course, but if you can confidently export an accurate CSV, that's a much better choice.Why? Autodiscovery processes are generally 5 percent to 10 percent inaccurate, regardless of product cost and complexity. That means you'll need to audit the results. So if you're comparing products and one is lacking autodiscovery, remember that not having a discovery engine isn't that big a deal.

Here's what really counts:

>> The interface on your monitoring system. If you're the only one dealing with the system, then go by personal preference. But if monitoring is handled by a centralized operations team--or if nontechnical folks are tasked with supporting users and monitoring network status--a no-frills CLI won't cut it. A GUI is easier to learn, especially if the monitor is used infrequently, and will make swinging back in the saddle and being productive easier. GUIs aren't perfect and won't cross all application borders, but they're much less taxing on the gray matter.

>> How the product alerts you when stuff breaks. There are two pieces to this: the way or ways in which you can be notified, and the sophistication of the software to notify you only when something is broken. Let's do the second first.

Large networks with lots of nodes being monitored will have lots of alerts. Many, like a load-balanced server going offline, are not earth-shatteringly important. Sure, someone cares, but do such alerts really require a BLT-interrupting page at lunchtime? We think not. Invoking this kind of subtlety requires the network-monitoring tool to know what is normal and, when something is not, to be able to gauge the importance based on the place or role in an overall service an affected device plays.

These distinctions are harder for network-monitoring software to discern than you might think; humanlike judgment requires some pretty fancy (read: expensive) software. If your network services are straightforward, event deduplication, downstream suppression and root-cause analysis aren't things you'll need.

Notification

A more practical consideration is to find out which types of notification the product supports. Sending an e-mail is important, and most tools will do this. Sending a page, while also widely supported, is more problematic. Some products can page only by dialing a phone number using an attached modem.

Better products provide tested scripts that understand the interactions that your beeper provider requires. If you're interested in using SMS as a notification method and ditching the beeper, check that the software supports SMS. It's more common in commercial products.You'll also likely find icons that change color based on the severity of a problem, with an associated sound or WAV file. Better products support the execution of an external script or program, for example, launching PerfMon and beginning to capture CPU performance.

Managing the process of alerting gets more difficult in very large network-management products and when lots of people need notification round the clock. But what's important for small implementations is to be able to set multiple paths for notifications to take. For example, you might want an e-mail sent and a WAV to play "Stars and Stripes Forever" when your Web server fails to respond.

But what if you're off-site, pondering curly fries or slaw for the BLT? The network-monitoring tool needs to have a trigger that says, "If the Web server is still failing after John Philip Sousa has finished serenading everyone in the office, set off a protesting vibrating pager dance just as a steaming plate-o-fries arrives." Make that a "to go" container.

Layer 2 Layout: Layer 2 Discovery Digs Deep

Keeping Time With Your NetworkIf you'd really like some basic network monitoring but aren't buying the simple part of SNMP, consider a monitoring ser- vice. There are plenty of them out there, but for the most part, they also cost plenty. At the low end, simple Web site monitoring providing simple "It's up/it's down" metrics can be had for as little as $7.00 per month per URL from a service such as 1stmonitor.com. This isn't going to tell you why the site is not working, of course. That kind of information is much more expensive, ranging into the thousands of dollars per month. For most or us who need straightforward, afford-able network monitoring, D.I.Y. is the only real choice.

Questions to ask providers

* What, if any, guarantees to assure your notification will the vendor offer? You're not likely to find human intervention or complex notification escalation at bargain basement prices.

* What is the cost per page/item monitored, and how frequently is that monitoring taking place? Every 5 minutes or every 30?

* Can they monitor securely behind your firewall? If so, how?0