Intrusion-Protection Systems

Two-and-a-Half Approaches

We found significant differences in approach and results among the participants in this still-developing category. Our invitation specified that each device must be a self-contained system able to identify network attacks and prevent them through its own action, rather than by sending commands to a firewall or other piece of network infrastructure. We also requested systems capable of handling the expected 400-Mbps flow through our test network core switches.

As it turned out, we tested using traffic moving across the core of the university's network, where flows averaged more than 600 Mbps, peaking at more than 800 Mbps with 180,000 to 250,000 simultaneous connections. Busy students. Of course, we wouldn't penalize entrants for not coping with conditions we hadn't told them to expect, but the larger flows did give us an off-the-record look at device capacity, revealing how the products handled a large amount of real network traffic with lots of live exploits and false positives. We also generated traffic with specific simulated attack types to see how successfully the devices stopped common exploits (for more on our test setup, see "How We Tested Network Intrusion-Prevention Products").

In the final analysis, two-and-a-half distinct factions emerged in this comparison:• The products from Fortinet, ISS, Juniper, Radware and TippingPoint primarily use signature patterns to determine if an attack is taking place.

• The Check Point and V-Secure products base their protection primarily, if not solely, on learning the characteristics of normal network traffic and using heuristics to protect against abnormal behaviors.

• The "half" refers to signature-based products built on open-source technology. Lucid's and SecurityMetrics' offerings, using applications such as Snort combined with proprietary console and management software, are more configurable than their proprietary brethren.

All can reset TCP connections and block, for varying lengths of time, new connections from proven attack sources. Some can rate-limit particular traffic streams, and Radware's product offers sophisticated traffic-shaping capabilities. Each strategy has strengths, but the winner and runners-up are concentrated in the more polished products of the signature-based category.

Although we were impressed with the field overall, no participant earned an "A." When Mike Fratto reviewed IPS products last year (see "NIP Attacks in the Bud," ID# 1417f2), this top grade was similarly elusive. We're not sure whether the products' immaturity or our higher expectations are the cause. But the maturity issue came into play when it came to how the products grouped in the scoring--those based on established IDS technologies scored higher.According to readers, the biggest concern with these devices is false positives, where legitimate network traffic could be blocked. In most cases, you'll want to stop high-volume and disruptive attacks, knowing that attacks on the fringes of your definitions will get past the IPS and be stopped by other network components. Because almost all IPSs are deployed inline, false positives are almost certain to generate more user complaints than attacks stopped by an application firewall.

In the first phase of testing, ISS' Proventia identified the majority of attacks confirmed by our IDS with very few false positives. TippingPoint's UnityOne tended to underwarn. This is in contrast to Radware's DefensePro, which was tougher than a celebrity bodyguard, treating virtually anything anomalous as a possible undesirable.

The FortiGate's signatures also discovered many of the attacks confirmed by our IDS, while flagging some activities that signature refinement would pass. While Juniper's NetScreen IDP identified a considerable amount of traffic as problematic in a default configuration, generating a high number of alerts that might be considered false positives, this product begs to be customized, with a toolset that will make the modification process quick for a security specialist.

Accuracy of the heuristics-based products depends on the device observing your network and developing statistical models for normal traffic, then looking for packets or exchanges that fall outside the norm. They start their deployments relatively loose to avoid false positives, then become more active and restrictive over time.

Fortinet's FortiGate and Juniper's NetScreen ended in a numerical tie, thanks to their consistently strong showing in critical areas. They balanced solid default performance and easy setup with rich functions for drilling into attack details and writing custom signatures. However, we award only one Editor's Choice per review, and Juniper's NetScreen has the edge. FortiGate is reasonably priced and has tons of good features and a well-made interface, but NetScreen is the most flexible and powerful IPS we tested. (All prices are list as configured for this test, with 400 Mbps as the target bandwidth and a single instance of the IPS engine.)There are systems designed to make intrusion prevention an automated and unobtrusive process--and then there's the NetScreen-IDP 1000. If your security staff includes someone with the know-how and desire to delve into every detail of an attack and who will be tasked with writing custom signatures to handle the specific requirements of your network, the IDP 1000 is your kind of IPS. The system has a solid, professional interface that lets those with expertise understand the context for attacks. What's more, it has the best facilities for custom signatures.

One example of the detail-oriented nature of this system is the device's window into the activity log. As incidents were displayed, we could mouse over an attack and receive details about the signature that triggered the response. Filter information regarding IP addresses of attack target and origin, along with time of attack, were well-laid out and useful.

Like many of the systems we tested, the IDP 1000 has multiple administrative interfaces. We ran into a bit of inconvenience between the interfaces when we were reconfiguring the system after the first testing phase. When it came time to change from sniffing to bridging mode (in which attack blocking takes place), there was no way to make the change in the dedicated management interface--we had to attach through the Web interface. We'd prefer to see each interface allow full configuration and management, rather than force you to remember which interface is used for particular tasks.

After the reconfiguration, we noted one interesting artifact: When we put the IDP 1000 inline and looked at a reporting screen, we were shown a large number of attacks in the previous 24 hours--unusual for a device that hadn't been switched on for three days. The results were valid, but something in the initial restart caused a timer to report badly. A restart cleared the problem.

When we began our generated traffic test, the NetScreen dropped a lot of traffic for legitimate reasons--if headers and ports don't match, it doesn't let the traffic through. The device reset connections and/or blocked traffic from specific IP addresses, depending on the type and severity of the triggering event. In the default configuration, this system reminded us more of an anomaly-detection (and rejection) box than signature matching.

During all of this activity, though, we could see excellent summary data, to which the system added context/ decode information. With one additional click, we were transported to an Ethereal page with the decode of the surrounding packets--invaluable for those who know what to do with the data.The signatures we received did not stop the embedded picture attack, but the IDP 1000 identified as problematic more traffic than any other system in the generated-streams testing. This device is expensive for intrusion prevention, but in the hands of a security expert the IDP 1000 provides a rich toolset and strong management.

NetScreen-IDP 1000, $49,995. Juniper Networks, (888) 586-4737, (408) 745-2000. www.juniper.net

Fortinet's FortiGate-3600 left us with mixed impressions during testing: Its rich feature set and very good user interface were offset by less-than-stellar performance at certain points in the testing. When the dust had settled, though, the FortiGate posted one of the top scores in this review, based on its manageability and ease of use, combined with enough features to please the most ardent configuration tweaker.

The FortiGate approach to security began before we connected a single management session. We configured a number of parameters using the front-panel switches, including a requirement that any configuration changes be authorized by a PIN in addition to the standard user name and password. When we connected to the FortiGate, it was through a standard Web browser--Java is not used, and Internet Explorer is not required. SSL is enabled. After beginning the session, we moved into a clearly laid out and logically organized user interface, providing ready access to the most complete set of features we saw in this review.

As a signature-based device, Fortinet takes the automated approach, in which new signatures are pushed to the device through the management system after being downloaded from Fortinet's signature service. An e-mail announcement is sent to a notification contact or list, and a one-click manual process is available for those who wish to override the automatic update feature. We would have liked a third option, in which signatures are automatically downloaded but administrative confirmation is required before they're installed in the device.If you need custom signatures, the facilities for writing them are included with the FortiGate. We found it relatively easy to add our own to the 600 signatures available at testing time. The interface for creating signatures is not as logical in layout as the rest of the interface, but it does follow the Fortinet approach to features--there are lots of them.

The FortiGate has four fiber SC interfaces, along with a pair of Gigabit Ethernet copper interfaces and a Fast Ethernet management interface. We could manage the Gigabit interfaces as individual network zones or combine them into larger zones, across which policies can be applied and enforced. The range of policies that can be created and deployed let us make the FortiGate behave like many security devices--from a standard firewall to a highly restrictive activity-prevention system.

The FortiGate-3600 is rated as a 4-gigabit device, and at simple levels of activity, the box easily handled traffic up to the 1-Gbps limit of our tests. However, as the rules we applied became more complex and more of them were put into place, the device's performance suffered. We found that with the system fully configured, the base latency peak with no device under test in the network was triple what it had been when the traffic flow went beyond 500 Mbps. There's nothing unexpected about this behavior, but you should keep it in mind when considering how to deploy this, or any, IPS.

Fortinet's signatures capably detected the expected range of attacks seen in the live-data section of our testing and stopped the attacks generated in our second testing phase, but there was one small disappointment in the default configuration: A signature for the Code Red worm was included with the signature set, but it was not activated by default. This is a sufficiently common attack to require inclusion in the default configuration. As with virtually all IPSs, tweak the signature and response base before taking the FortiGate live; fortunately, Fortinet makes the customization process fairly easy.

The FortiGate-3600's price is a relative bargain in this market, though you must budget for a maintenance agreement to continue signature updates after the relatively short 90-day term. That's a small issue, though, as were all of FortiGate's shortcomings. It's not perfect, but it's one of the best signature-based IPSs we tested.

FortiGate-3600 Antivirus Firewall, $29,995. Fortinet, (866) 868-3678, (408) 235-7700. www.fortinet.comG1000-400 ISS provides a full set of features in a full set of products with its Proventia G1000-400, SiteProtector software and SiteProtector Management Console. The hardware and software work together to provide network protection, though you'll work a bit harder to get at them than you will with either Fortinet's or TippingPoint's products.

ISS recommends that the SiteProtector and SiteProtector Management Console software components run on separate systems. We agree--we tried running the two apps on a single server. Although it was a well-configured server (dual Xeon processors and plenty of RAM), the management console's performance was noticeably slow in several situations, particularly when we were trying to generate reports based on sizable log files. SiteProtector Management Console is written in Java--no doubt because of the hardware requirements--but you should be aware of the load the software places on a server.

Once we got into the interface, though, we had access to many features, and could view incident reports and the packets that generated activity. A variety of predesigned reports can be run ad hoc or on a regularly scheduled basis for managers and admins. Security specialists will be more interested in protocol-decoding capabilities, and ISS does a good job here, though it doesn't present the entire decoded packet. ISS says an upcoming software update will provide this ability.

Between the two extremes of management reports and packet decodes are many ways of grouping attacks and viewing the activities of the IPS. The richness of options makes the degree of learning difficulty a bit higher than with rivals, but the result is an ability to discern a great deal about how your network is being targeted and by what means--valuable information as you look to enhance your security. The Dashboard features of the Management Console gave us access to reports, charts and other graphics showing what the device has been doing while in operation. The console (not the sensor) really wants to be on the Internet--it went out looking for the ISS Web site to find the information used to explain attacks that are stopped or noted in passing.

The detection engine is also impressive. In our live data testing, ISS identified the majority of attacks without blocking much legitimate traffic. During the generated traffic testing, the G1000-400 stopped the Code Red worm with signatures and responses defined as a default event within the interface. The management console showed the stopped attack as an event rather than a standard attack--all the individual facts of the event were correctly reported, but we were fascinated by the bin into which the attack was placed. From a security standpoint, we found a solid level of paranoia built into the system; for example, when we started the sensor, it wouldn't pass any traffic. Once we configured interfaces and zones, we found that attacks were properly identified and stopped. The rule was nice and tight, too, allowing legitimate traffic through, though similar in many respects to banned traffic. Other traffic passed without noticeable latency being introduced at any traffic level up to the 400-Mbps rated throughput and beyond.

There's a plug-in for using the ISS vulnerability-assessment scanner as part of the total management interface--useful for organizations looking to build integrated security capabilities. In the final analysis, there are a lot of nice touches in this reasonably priced system. If you need a constant stream of reports for management, or if you simply need an IPS with very solid reporting for your own analysis, the Proventia appliance is a sound choice.

Proventia Intrusion Prevention Appliance G1000-400, $29,314 (includes tech support, updates and advanced exchange; unlimited SiteProtector console costs are built into the appliance price). Internet Security Systems, (800) 776-2362, (404) 236-2600. www.iss.netTippingPoint's UnityOne-1200 Intrusion Prevention System is the best unit we saw for out-of-the-box "set it and forget it" intrusion prevention. If you want an appliance that will handle a lot of traffic with solid protection while insulating your network admins from the nitty-gritty details of the IPS, the UnityOne is for you. But if you take a serious hands-on approach to tweaking an IPS, there are some portions of the interface that will give you pause.

TippingPoint starts with a clean user interface that didn't give us too many places to go looking for things. It almost feels like there aren't enough things to do, but that might be related to the number of functions enabled out of the box, such as workable initial configurations for signature use, response and reporting. Setup was quick and easy, but some daily administration items are hidden--TippingPoint made us jump through hoops to get raw data to verify which packets triggered events, for example, or for forensics purposes.

TippingPoint says it designed UnityOne to always be deployed inline; the company had serious reservations about the first phase of our testing. The product handled the optical tap testing phase quite well, though, easily dealing with the high bandwidth and consistently picking up attacks. When we moved to the inline phase, the device's performance was in keeping with its inline deployment model, striving to minimize false positives.

UnityOne successfully stopped both the Code Red and JPG exploits in the second testing phase. When we looked at its performance results, we were puzzled by some jitter--latency increased, but individual packet latency varied widely because of the nature of our test traffic. Then we realized we had presented a very nearly worst-case scenario--there was a lot of strange traffic that the UnityOne was performing deep inspection on before deciding that it didn't present a threat and allowing it to pass. In a normal network, performance shouldn't be a concern.

TippingPoint includes an optical bypass device to ensure network continuity if power to the UnityOne is lost. The bypass worked as advertised, continuing to pass data when we pulled the plug on the device.

The UnityOne was the most expensive system we tested, though some of the purchase price might be recouped in not having to hire security specialists to deploy the box. For a complex security device, the UnityOne was remarkably easy to install, configure and get to a useful state. If it had offered easier access to the dirty details of attacks and exploits, the UnityOne might well have been at the top of the heap.

UnityOne-1200 Intrusion Prevention System, $64,995 (includes four ports). TippingPoint Technologies, (888) 648-9663, (512) 681-8000. www.tippingpoint.comSecurityMetrics entered the IPS market with a system based on Linux, Snort, Nessus and other open-source software joined with a custom integration and management wrapper. The Model 60 shares basic functionality with the ipAngel (they have similar foundations).

Unfortunately, SecurityMetrics' system has some of the rough edges common to early IPS products--though there is plenty of promise beneath. It came in with the lowest price in this review, just $14,999.

The Model 60 didn't do well in our live traffic testing scenario, in which 650-Mbps traffic peaks were common--no surprise, as the unit is rated for only 200 Mbps. When traffic was throttled down to 200 Mbps, the SecurityMetrics appliance easily handled the full flow. In fact, the Model 60 handled traffic up to our projected 400 Mbps, though there was considerable jitter in the flow at that point.

Because Snort is an integral component of the SecurityMetrics package, the full packet decode was readily available when intrusions were detected. The Model 60's interface let us get solid reports of activity without having to drop down into the basic software interface on a regular basis, and we could configure the system to handle almost any set of circumstances. The only question is how much time and expertise you'll have to invest in the configuration.

In our generated traffic testing, the system had trouble with the Code Red traffic--it triggered on the event, but with the wrong signature. The device was configured to detect Code Red, but it doesn't pick it up under that name, though the attack was, in fact, stopped. The Model 60 didn't stop the "JPG picture" attack, though this was a much newer attack at testing time.

If your needs are more modest than the large-enterprise model we used in our testing, the Model 60's bandwidth limitations should be of no concern. Most of our problems with functionality concerned immaturity in the interface.

SecurityMetrics Security Appliance Model 60, $14,999 as tested; Model 10 starts at $5,999. SecurityMetrics, (877) 311-4400, (801) 724-9600. www.securitymetrics.comRadware's DefensePro is a big-bandwidth box with strong features. It handled all the traffic we threw at it, but its management and analysis interface is less intuitive than rivals'. We spent far more time in Radware's "industry-familiar" CLI (the interface looks as much like Cisco's IOS as the lawyers will allow) than in any other system's CLI. Add in the fact that we got to know four separate Radware boxes, and we wound up with a high-performance product that finished in the middle of the pack.

Let's get the four boxes out of the way first. We're not sure what happened--nor are the Radware engineers--but three consecutive DefensePro devices didn't like our lab. Problems ranged from poor bandwidth handling to a refusal to recognize new attacks. The fourth box settled in and performed well.

It's important to note that the DefensePro works in ways that differ substantially from most of the other devices we looked at. Where most IPSs will reset connections, block external IP addresses and quarantine internal addresses, Radware's system adds traffic shaping and bandwidth limiting to make for a more sophisticated set of responses to attacks. DoS attacks, for example, can be limited to a small portion of your total bandwidth, minimizing the impact while letting legitimate traffic from the offending network (or server) continue.

Radware did well in terms of the bandwidth and latency numbers, adding no meaningful latency even as the number of signatures, rules and responses increased. During our generated-traffic tests, the DefensePro showed attacks even when we weren't sending any. The random data we were using to fill up some of the noise packets meant that there were occasional mismatches among headers, ports and packet contents. The DefensePro did successfully stop all our bad traffic and identified the legitimate attacks seen in the live data-testing phase.

The default settings from Radware were restrictive, and we ended up tweaking considerably (as you will with any product of this type) to limit the number of positive responses.

DefensePro's $48,000 list price places it in the top tier of devices we tested, but the price isn't out of line considering the overall capability of the system. The console is fully functional, though not as tweakable as those from ISS and Check Point. If you want the ability to shape traffic that contains attacks, rather than merely turn it on and off, Radware provides a sophisticated solution.

DefensePro AS-III/SME, $48,000. Radware, (888) 234-5763, (201) 512-9771. www.radware.comThe InterSpect 610 was one of several devices built on a Dell 1U server, in this case a dual-CPU box. Check Point dedicates one CPU to each of its interfaces, making for some performance improvements where network traffic flows are symmetrical. The InterSpect 610 lived up to our performance expectations, with no meaningful latency introduced, and it offers an easy-to-understand, mature user interface for configuring and administering its functions.

Most of our interaction with the InterSpect 610 took place through the dedicated client software. The first thing we saw, though, was the Web interface, which let us download the client software. Check Point provides a display of the encryption public key fingerprint so you can verify that no man-in-the-middle attack is under way--that's reassuring if you're installing the client through a public network segment. This is one of the little details that speaks to the maturity of the design.

The uncluttered interface will go too far for some users, though. We found ourselves wanting more specifics on attacks and real-time view flexibility at several points. There were only a few straightforward ways of looking at data, and we missed having a variety of views and drill-downs into, for example, events count, newest events, severity and appropriate responses. The standard reports of activity, in particular, should be fine for high-level admins, but will lead most security specialists to ask more questions. On the plus side, InterSpect required us to learn only one interface; we could then access all the functions available.

The InterSpect 610's coverage is based largely on heuristics. These devices continue to refine their detection and response characteristics over the life of their deployment, so it's highly unlikely that any limited-duration test will showcase all their capabilities. And yet, the InterSpect performed well. There was a bit of labor involved as we set up the individual interfaces and defined zones in which particular types of behavior were expected. Once that quick process was done, the InterSpect recognized worms and exploits in both our live feed and generated traffic testing phases.

InterSpect began its deployment relatively loose, but it's designed to become more accurate as it learns your network. You can speed deployment by manually adjusting the settings, an easy process using the Check Point interface. After looking at reports from the first portions of our testing, we did tweak settings so that the InterSpect was more active in reporting worm detection. The facilities for writing your own signatures are built into the system's software, with an interface that's consistent with the product's straightforward nature.

The InterSpect 610's price places it in the middle of the pack. If your network is relatively stable, so that the heuristic learning process can become fully involved with normal traffic, it's a solid, reasonably priced option.

Check Point InterSpect 610, $36,000. Check Point Software Technologies, (800) 429-4391, (650) 628-2000. www.checkpoint.comThe ipAngel is one of two systems we tested that make use of the open-source software available to run on Linux. In this case, Lucid Security has used Red Hat Linux as the foundation for a complete IPS system. Like SecurityMetrics' Model 60, the ipAngel wraps a useful central interface around a variety of programs, such as Nessus, Snort and iptables, integrating the pieces in a single IPS server. Lucid has done a good job of joining functions, and at a very competitive price. Unfortunately, Lucid's implementation has a few quirks.

The first major problem is the requirement that the ipAngel attach to the Internet to validate its license key and update its signature files. This is a risk many network pros prefer not to take. It would be nice if the risk were accompanied by a simple registration process, but Lucid made it more involved than any of the other products we tested.

The ipAngel can be set up to recognize and respond to a wide variety of threats, including all the generated threats we used in our test scenarios. But we ran into some difficulties with setting up the device and seeing precisely ipAngel's activities. While the Lucid interface provides a measure of insulation between the user and Linux, there are elements of customization that only Linux experts could perform. In our case, we managed to create a group of firewall rules that kept us from accessing the ipAngel through the usual interface! A quick trip to the Linux command line and modifications to iptables restored management interface access.

Our problem with reporting started with an apparent inability to write events to a external syslog server or report events based on SNMP traps. These aren't fatal flaws, but they do make it more difficult to incorporate the ipAngel into a larger security infrastructure. Lucid also made us work harder to see information on a real-time basis than with SecurityMetrics, though we did eventually get all the information we needed out of the management interface.

The ipAngel counts as one of the bargains in this bunch. The device has a good legitimate attack hit rate while not flagging a high number of false positives, using a foundation that has been vetted by a large number of users as part of the open-source process. In addition, the interface has a mature feeling. If your network staff eats and breathes Linux, you might want an ipAngel watching over your network.

ipAngel X3 AVS-400, $18,000 ($15,000 hardware; $3,000 annual subscription). Lucid Security Corp., (215) 371-3300. www.lucidsecurity.com

V-Secure is a confounding system--it does several things very well and has interfaces that left us scratching our heads.The V-1000 was the only product based solely on heuristics. Its monitoring and blocking model is very creative and can be effective, though not without problems. V-Secure operates on a feedback-loop system, where the box sees an attack, tries a filter and then quickly reacts on feedback to the success of the filter. But the product is almost entirely dependent on anomaly detection--and on catching most attacks during the reconnaissance phase. If someone uses Google to find a vulnerable server, then launches a direct attack against a known vulnerability, an anomaly-detection system is unlikely to detect the traffic pattern. This shows the approach's limitation--it really takes a pattern-matching engine to deal with direct, no-recon attacks. With that said, our testing and analysis indicate that V-Secure might well be the best system for detecting and stopping DoS attacks--it reacted so quickly and shut them down so thoroughly that they had practically no impact on our network.

In our performance tests, the V-Secure carried 400 Mbps easily. When we punched up the traffic load, it slowed considerably as it neared 1,000 Mbps, finally hitting its limit and rolling over. In addition, it didn't like private network addresses being routed into its public-side interface and complained mightily about them. Whether because of that, or because we didn't repeat the tests often enough to train the heuristics, V-Secure did not stop Code Red traffic or our JPG exploit. In the first phase of testing, though, V-Secure recognized many of the attacks launched against the network.

The primary user interface is laid out logically, and while it's not the most intuitive interface we saw, it doesn't work against you. We could find attacks and see what triggered responses. Our real problem came in the multiplicity of interfaces available for different functions. There are no fewer than four interfaces for administrators, managers, executives, technicians and, quite possibly, the night-shift cleaning crew. None relate to any other, and trying to move among them to get useful information is an adventure.

The V-1000 is a box with a ton of promise, but at $45,000, there should be more delivery than promise. Another generation of development could turn this system into a truly outstanding device.

V-Secure V-1000, $45,000. V-Secure Technologies, (201) 291-2845. www.v-secure.com

Curtis Franklin jr. is a senior technology editor for Network Computing and Secure Enterprise. He has been writing about the computer and network industries since 1985. Write to him at [email protected].

Jordan Wiens is a security engineer at the University of Florida, Gainesville. Write to him at [email protected].In theory, intrusion-prevention systems add a layer of protection to networks by denying suspicious activity based on preset rules. We teamed up with the University of Florida at Gainesville to put nine IPSs to the test. After sending streams of background noise, we tried to slip attack traffic past Check Point Software Technologies' InterSpect 610, Fortinet's FortiGate-3600, Internet Security Systems' Proventia Intrusion Prevention Appliance G1000-400, Juniper Networks' NetScreen-IDP 1000, Lucid Security Corp.'s ipAngel X3 AVS-400, Radware's DefensePro AS-III/SME, SecurityMetrics' Security Appliance Model 60, TippingPoint Technology's UnityOne-1200 and V-Secure Technology's V-1000.

The vendors used a variety of methods to protect our test network, but we preferred the more polished products that depended on a signature-based approach. Fortinet's FortiGate-3600 and Juniper's NetScreen-IDP 1000 placed well in all our scoring criteria and ended in a numerical tie. However, while we awarded Juniper's mature, powerful and flexible NetScreen-IDP our Editor's Choice this time, the company should keep a close eye on its competitors in this maturing space--we will.

We conducted testing in two phases. The primary phase made use of the University of Florida's full Internet and Internet2 bandwidth in a passive configuration to evaluate how the products handled a large amount of real-world alerts and noise. Net Optics provided an eight-port regenerative optical tap that we used to bring a stream of traffic into the lab, and existing span ports mirroring traffic were also available.

The fiber tap has the advantage of being able to simulate two sides of a conversation to an IPS (intrusion-prevention system). While a span port dumps both sides of a conversation on one interface, a fiber tap separates the data into two distinct sides of conversation; this is how the device would actually function, making a fiber tap the best way to test an IPS that would normally run inline.

Devices were connected via an isolated Cisco 2940 switch or crossover cable to a management and reporting station (the switch was necessary only in products with at least three distinct components on three different platforms). We used VMWare for hosts requiring an additional management package. When Web interfaces were needed, we used Mozilla on Red Hat Linux or Internet Explorer under Windows inside of VMWare.In the second phase of testing, we evaluated the products in true inline mode. Ixia provided a 1600T mainframe system configured with two blades, one providing four Gigabit Ethernet interfaces, the other eight 100Base-T interfaces. Traffic was generated using IxChariot software on the Ixia hardware and a Hewlett-Packard laptop computer running a variety of lab-developed custom scripts. With Ixia's assistance, we developed a test configuration to send background traffic (composed of real-world protocols) at various rates through the IPS while simultaneously running simulated attacks. These consisted of IE browser exploits: Web pages being transferred that attempted this attack, taken from a real Web page found in the wild; nmap-like scanning (sequential Port 80 requests); and Code Red IIS exploit requests.

Traffic generated by Ixia was sent through two Cisco 3750 routers that were initially connected to track baseline traffic patterns. The IPS products were placed between the two routers to test their inline functionality.

One artifact of the traffic generated by the in-lab equipment: While TCP ports were used to simulate different protocols, the data within the packets did not necessarily match the protocol. Many of the IPS products generated alerts that the traffic was not legitimate protocol traffic for the ports it was on and then blocked the suspect traffic. The bandwidth figures in this phase are therefore not as accurate as an actual deployment with real-world traffic in that the devices potentially saw a much higher rate of errors (causing more processing and alerting) than on a typical network of equivalent bandwidth.

All Network Computing product reviews are conducted by current or former IT professionals in our own Real-World Labs®, according to our own test criteria. Vendor involvement is limited to assistance in configuration and troubleshooting. Network Computing schedules reviews based solely on our editorial judgment of reader needs, and we conduct tests and publish results without vendor influence.

R E V I E W

Network Intrusion Prevention Systems

Sorry,
your browser
is not Java
enabled

Welcome to NETWORK COMPUTING's Interactive Report Card, v2. To launch it, click on the Interactive Report Card ® icon above. The program components take a few moments to load.

Once launched, enter your own product feature weights and click the Recalc button. The Interactive Report Card ® will re-sort (and re-grade!) the products based on the new category weights you entered.

Click here for more information about our Interactive Report Card ®.

0