Policy Enforcers

• Monitor compliance against a defined configuration across multiple administrative domains and OSs from a single management console;

• Query systems for configuration, user accounts, access controls, and patch and service pack levels;

• Provide multilevel reports on computer configurations, from detailed technical information to high-level roll-ups; and

• Optionally, fix discovered problems proactively.

With this model in mind we gathered seven security policy monitors--BindView Development Corp.'s bv-Control 7.2 and Policy Operation Center 4.2, Computer Associates International's eTrust Policy Compliance 7.4, Configuresoft's Enterprise Configuration Manager 4.0 and Security Update Manager 2.0, NetIQ's VigilEnt Security Manager 4.0, Pedestal Software's SecurityExpressions 3.0, PoliVec's Security Automation Suite (Builder 2.6, Scanner 3.5 and Enforcer 1.1), and Symantec's Enterprise Security Manager 5.5--in our Syracuse University Real-World Labs®. Xacta Corp. declined, saying its product was between versions, and Tivoli did not respond to our invitation.We tested these products on our production and test servers and desktops, which run a mix of Microsoft Windows NT 4.0, Windows 2000 Pro, Server and Windows XP, Sun Microsystems Solaris 2.7 and 2.8, and Red Hat 7.3 and 8.0--in all, more than 100 machines in various states of configuration and patch levels.

What We Want

We looked to create compliance checks from our existing policy. Compliance checks can be as simple as testing for a registry key value or a Windows 2000 Group Policy Object setting or as complicated as checking the user/ group rights to directories and files across all platforms. All the products we tested let us create complicated compliance checks: For example, we could check the audit configuration on a subset of computers. What counted here was the ease of defining those checks--for skilled administrators, time is big money. Perks like context-sensitive drop-down selections, feature definitions, sample compliance checks and complex expression building are pluses. BindView and NetIQ nailed this area.

Once compliance checks are made, the generated reports must be informative and customized for your audiences. For example, executive-level reports don't need to contain technical details, and technical reports aren't enhanced by roll-ups. Nearly all the products we tested can export reports to various formats or databases--a huge plus. The products from Configuresoft and Symantec shine in reporting because of their overall readability, level of detail and ability to create reports with varying levels of granularity.

PoliVec's and NetIQ's security-policy-generation applications have security policy templates that can be customized and distributed to users for review and signatures. In addition, policy statements that are enforceable on desktops and servers, such as password requirements and group security settings, can be generated automatically into a template that is used to check compliance on target computers. BindView's Policy Operations Center creates policies but doesn't export compliance checks.Although reports are important, ad hoc queries are key when you're figuring out what can be reported and for determining the state of some network features. Like policy building, defining queries should be relatively simple. The more targeted and complex the queries, the better the score. For example, when we asked for a list of users who hadn't logged in for 30 days, we wanted just those accounts, not a report with all accounts listing a date or an interval showing last login.

Rounding out our requirements were OS support and remediation. Many heterogeneous networks run only Windows and common Unix-based OSs, like Solaris and Linux, but if you're dealing with more exotic species like IBM's AIX, OS/390, AS/400 or VMS, you need a policy-monitoring system that can support all your platforms.

The value of remediation depends on the role security personnel play in your organization. All the products tested could make some changes to target configurations. But only Configuresoft's and Pedestal's products could push out Windows security patches. Service-pack installation was supported by Pedestal's SecurityExpresssions through a customized script. However, in many organizations, operations or desktop/server staff control the deployment of patches, service packs and configuration options--the authority to make changes or deploy software crosses boundaries, and those boundaries will have to be defined before using remediation features.

All the products that required agents offered remote-deployment and silent-installation packages, which we could distribute through a login script or via a desktop-management application. While the Windows desktop administrator in us favors the agentless monitoring programs offered by Pedestal, PoliVec and BindView, the downside is coordinating the domain or local logins for each target. Agents run as a system account and aren't prone to losing communications because a user account changes its password or the computer is off-network.

Finally, cost is a sensitive issue, so we weighted price heavily in our scoring. No matter how you slice it, we couldn't justify spending more than $200,000 on 1,000 licenses. Even $150,000 is a stretch.Because there is no way to determine a reliable street price for these products, we asked for list price based on two scenarios and calculated grades based on this information. Of course, nobody pays list, so your cost will vary depending on your negotiating power. We scored on the assumption that discounts would be proportional. Note also that pricing for these products has a linear dimension because it's based on a per-system model.

Cost aside, we were pleased with all our entries. To one degree or another, each provided compliance monitoring and ad hoc query functionality. Reporting varied greatly across the board, however, as did OS and application support. BindView's solution captured our Editor's Choice by virtue of its granular policy and query definition and decent reporting, all at a reasonable price. Speaking of price, Pedestal really shone in this area and scored our Best Value award.Remember the model airplanes you built in your youth? There were a lot of parts to put together with glue that made you dizzy, but the final result was worth it. Installing bv-Control brought back those memories. Luckily, we had to install it only once.

The product has an interesting access-control model where administrators are added using their local or domain accounts. Once in bv-Control, users are granted access rights to specific portions of the application. By default, administrators in bv-Control don't have the right to make configuration changes on managed computers using ActiveAdmin. That right has to be granted specifically. The upshot: You decide who has read and write access to target computers.

For bv-Control to interact with monitored computers, a credential database is created that stores user accounts and passwords for the target domain or computer. We created a domain administrator, called "bindview," in all our Windows domains, then added that account to the bv-Control Credential database. For our Unix computers, we added a local "bindview" account to each computer (we don't run NIS or NIS+) with root group privileges.

Finally we added snap-ins for our target applications, such as Windows desktops and servers, Unix computers, and Active Directory. Each managed application or OS required a specific snap-in, and during the installation phase, the credential database was specified and a query engine assigned. The query engine carried gathered data and could be installed anywhere, provided the management station could contact it. For example, a query engine might be installed in a remote office to limit the amount of bandwidth used across the WAN. Windows computers are agentless, but the Unix ones aren't--and the Unix agent installation was painless.We were pleased that bv-Control supports a wide variety of OSs; we tested with Windows 2000 Pro and Server, Red Hat Linux 7.3 and Solaris 8--the latter two required agents. Within a few minutes of a successful install we were happily generating reports using the many predefined formats and figuring out ad hoc queries. The reports were readable, though not as informative as those generated by Symantec's Enterprise Security Manager or Configuresoft's Security Update Manager.

Reports could be run against all the computers in the domain at this point, but sometimes we wanted to run reports against a subset. "Scopes" define a subset of targets and act as a grouping mechanism. Computers can be added manually to a scope--an easy process--but we preferred Configuresoft's approach of defining a scope according to properties, so computers are automatically added per group.

Generating basic reports was painless using one of the canned formats: We set the scope and ran the report; bv-Control remotely queried the target computers and gathered the data. Targets that were unreachable showed up as errors. We liked that we could export reports to a variety of formats and data sources. And we customized several reports with the scope and report format and saved to a personal or shared folder.

Building custom reports and ad hoc queries was straightforward. For example, we wanted to see which files and directories allowed the "everyone" group full control. We created an assessment and selected the information we wanted in the report, like domain/workgroup name, machine name, directory name and where permissions matched "full control for everyone." Then we created a filter so that we would see only true matches. We could search for anything in the file descriptor.

Although bv-Control doesn't have an internal scheduler, any scheduler, including Microsoft's, can be used to run reports. We created a task list (a list of reports to be run) that included reports that discovered user accounts that were due to expire in 15 days, had not logged in for 30 days, or had been locked out. We defined the credential database that would be used and the scope of the reports, and we defined the report format. Then, in Microsoft's Scheduler, we created the task, which called a bv-Control command-line program, and passed the task list as an argument.Policy Operations Center is a hosted service that let us create a written security policy and distribute it to our end users. Several templates are available, or we could import our own policy. End users log into the site, read the policy, and acknowledge that they read it. Unfortunately, when we went to the selected URL in the guise of an end user, we were prompted to supply only a first and last name and an e-mail address. No authentication is required, and to make matters worse, there isn't a way to compare a list of users who should read the policy against a list of users who did read the policy. BindView told us that this feature will be available in a future release. In our opinion, for thirty-five grand, user authentication and tracking should be standard.

Coming in on the low side of the middle tier, cost-wise, bv-Control has a lot of features for a good price. If we add in the $35,000 for a one-year subscription to Policy Operations Center, the total jumps to the high end of the middle-tier pricing--$171,858 for the first year.

bv-Control 7.2 and Policy Operations Center 4.2, BindView Corp., (800) 813-5869. www.bindview.com

Configuresoft has some useful and unique features, but a big drawback is its Windows-centric focus. Sure, Windows has won the desktop, but as we all know, there's a whole lot of Unix and yes, NetWare, deployed. Enterprise Configuration Manager (ECM) provides the base functionality for device discovery, management and reporting, while Security Update Manager (SUM), which is licensed separately, provides patch reporting and updating. ECM is a powerful monitoring tool. That power comes at a price, though--the system is difficult to learn--but if you're an all Windows shop, ECM and SUM combined is a strong choice.

Unique to the products tested, SUM provides detailed patch and service-pack discovery and can deploy patches (though not service packs) to end systems. More important, SUM will display any dependencies that must be satisfied prior to the deployment of patches, such as a service-pack level. We ran SUM against our test computers and discovered many missing patches. We selected all the machines that needed patching and deployed everything in one fell swoop. SUM downloaded the patches from Microsoft's site and installed each one successfully.

ECM runs data collection through DCOM (Distributed Component Object Model) agents on target hosts and dumps collected data back to a database. All report generation is run against the database, not live on the hosts, so make very sure your database is current before running reports. Luckily, ECM can collect subsets of data as needed. Further, careful hardware planning is needed for the database: In our 100 host test bed, Configuresoft recommended a SQL server with dual PIII CPUs, 2 GB of RAM, RAID 3 and a 27-GB database partition. That's beefy.Lesson learned: ECM is really just a fancy front end for SQL queries. Once we realized that, many features made sense. For example, hosts can be placed into multiple groups automatically based on features/functions discovered on the hosts. We created a primary domain controller group, an FTP server group and an Exchange server group. We defined the filters for groups based on the features of each, and the groups were populated automatically. In reality, when we selected a group, ECM issued a SQL select command--the filter is just a "where" clause specifying the relevant records.

ECM comes with numerous predefined reports as well as an Explorer-like interface that we expanded node by node to show the targets that matched our criteria. Building reports takes a bit of work because of the all the data available. Also, many of the selections have drop-down lists derived from discovered data. Much of the learning process involves understanding where specific device information is kept. As with Pedestal's product, external programs can be launched on the targets through Visual Basic scripts to do custom discovery.

Enterprise Security Manager 4.0 with Security Update Manager 2.0, Configuresoft, (719) 447-4600. www.configuresoft.com

SecurityExpressions was deceptively simple to use compared with the other products we tested--and that's a good thing, provided you stick to the sample scripts and reports. SecurityExpressions, like the products from BindView and PoliVec, doesn't require agents for Windows computers, though agents are available for querying hosts on remote networks. SecurityExpressions is unique among the products we tested, however, in that it doesn't require that you install an agent on Unix hosts--it just needs an account with root privileges and SSH for secure communications.

Reporting and policy definition are top notch, though custom reports and ad hoc queries are a bit difficult to define. SecurityExpressions can install both patches and service packs through custom scripts. Of course, the price is by far the lowest of the products we tested, which, depending on your needs, may offset the costs associated with building custom reports.

SecurityExpressions splits the tasks of policy monitoring and policy enforcement by providing a Web interface, where we could run reports but not make changes to end systems or the SecurityExpressions application. Each report is contained in an individual SIF file: There's no global configuration for computer discovery and grouping, so we had to repeat basic configurations for each individual report. We exported the SIF files as policy guideline documents, which provided high-level descriptions of each check, or as policy standards where a specific configuration setting is detailed. Both reports are appropriate for administrators, not for general users.Running reports was a snap. We took a predefined report that discovered the patch levels on the destination computers and defined the domain credentials used to authenticate to each target. We saved the report file and ran it. The resulting report showed the missing patches and service packs; we then used this information to bring systems up to code.

Configuration reports offered a bit more detail and showed the defined settings as "OK" or "Not OK." There is a wizard for creating custom reports. However, we found that you have to know what you're looking for and the required options. Some selections are context-sensitive, but many aren't. However, we caught on quickly, so after gaining some familiarity and experience, most administrators should be able to build reports just fine.

SecurityExpressions is ideal for shops with more chops than cash--though it is more difficult to use out of the box, once you get the hang of it you'll enjoy robust reporting and scripting for a bargain price.

SecurityExpressions 3.0, Pedestal Software, (888) 664-7174. www.pedestalsoftware.com

Enterprise Security Manager (ESM) excels on reporting--the level of detail is certainly on par with that of Configuresoft ECM. In fact, the reports are so well designed that we needed to do very little customization. Roll-ups down to technical details are available within a few clicks. Unfortunately, ESM subjected us to some deployment difficulties, and its pricing places it at the high end of the spectrum.

ESM uses agents to gather data from remote hosts, and the agents can be centrally deployed--in theory, anyway. The process is somewhat convoluted, however, requiring that you build a temporary share on your management station and install a remote update agent on the targets. The manager should then instruct the remote update services to connect to the share on the management station to install the software and register with the manager; once successful, the remote update service will be removed from the target, and the share is also removed. We say should because we couldn't get any of our targets to connect back to the installation share on the manager, and Symantec couldn't figure out a fix. There is a silent installer, however, that we used with success.We used predefined reports to get a view of our network. For the roll-ups, ESM scores and totals misconfigurations for a measure of risk or vulnerability. While good for making generalizations about population status, the scoring is somewhat arbitrary. In this case it's in the details, where the devil resides, that ESM shines. The detail reports provided easy-to-understand information about each issue, how it was resolved, and sometimes even potential difficulties that may result.

Building custom reports and ad hoc queries, while possible, isn't as straightforward as we would have liked. Again, with familiarity and experience, customization became easier, but getting to that point took time.

Finally, many fixes can be applied, but ESM doesn't support remote patch deployment.

Symantec Enterprise Security Manager 5.5, Symantec Corp., (800) 441-7234. www.symantec.com PoliVec offers three loosely integrated products--Scanner, Enforcer and Builder--that do the work of other vendors' single products. This approach does let you purchase only those components you need, but the integration is not seamless.

Scanner discovers the configuration of network hosts and is agentless for Windows targets. Enforcer allows configuration changes to be deployed to target systems, using agents. In the case of Unix systems, Enforcer can also discover configurations. Builder creates from templates security polices that can be distributed and read by end users. Policies can also be exported as XML documents and imported into Scanner and Enforcer.Builder is a breeze to use--simply run through the wizard and select the policy statements to include in the final policy. Icons show which items are used in Scanner or Enforcer and often have configurable items. For example, we wanted a strict password policy, which we defined in Builder and exported to Scanner and Enforcer. Builder offers lots of explanatory text about policy statements, and each statement could be annotated and customized.

Scanner takes policies--predefined, imported from Builder or defined in Scanner--runs through the selected target, and reports back. Scanner is focused on comparing targets to configurations and lacks some of the robust ad hoc querying capabilities found in other products.

Enforcer is an automated monitoring tool that takes an implementation standard derived from Builder and checks for compliance. Alerts are generated on non-compliant items, and based on defined intervals, we escalated notifications on unhandled exceptions. Unfortunately, however, Enforcer kept collapsing due to corrupt keys, and PoliVec could neither replicate nor solve the problem. We, too, are stumped.

PoliVec Security Policy Automation Suite, PoliVec, (866) 765-4832. www.polivec.com

VigilEnt Security Manager (VSM) is a mixed bag of good and lame features at an expensive price. As with CA's product, the cost bump is in the server space. VigilEnt Policy Manager (VPM) is similar to PoliVec Builder in that policies can be developed and used as templates against target computers. Although VSM requires agents on target systems, proxy agents can scan up to a recommended maximum of 50 targets. Each target still uses a license, but you save the problem of deploying agents everywhere. Agents can be installed remotely; however, we ran into weird problems--the agents would install and run and then the service would shut down and issue a Dr Watson. Neither we nor NetIQ could determine the cause.

Reports are detailed, and we found customizing existing reports and creating new ones no more or less difficult than with other products we tested. We did, however, have difficulties limiting the data that was returned. For example, we wanted a report that listed only accounts that could act as part of the OS, but no dice. Could we do this with the products from Computer Associates and Configuresoft? Why, yes, we could. Applying a filter to a report isn't the same as generating the desired report automatically.We build ad hoc queries, and VSM made interactive queries easy to make by letting us define a variable name for a parameter. The value is requested at run time. For example, we created a query that showed all the owners of files within a file system. Then we specified the directory that should be used as the search root. As for remediation, VSM was one of the weakest products we tested--it could manipulate only user objects.

NetIQ VigilEnt Security Manager 4.0, NetIQ Corp., (888) 323-6768, (408) 856-3000. www.netiq.com CA's eTrust Policy Compliance (EPC) is loaded with useful features and, function by function, competes right on with our top contenders. It's a good tool for monitoring and remediation--if you can afford it. We simply can't recommend it, though, at the price of $470,000 for 1,000 hosts. The bulk of the cost comes down to tiered pricing, where bottom-tier servers start at $2,000 per agent; even desktop licenses are high, at $200 a pop.

EPC's reporting and remediation offered no surprises and are on par with bv-Control and SecurityExpressions. We did run into an issue where the silent agent installer failed to authenticate to the management server. CA sent us an update to fix the problem. Once the agents were installed, we were in business. Building and running queries was a simple matter of following some wizards. Same with fixing config issues. Unfortunately, however, fixes can be applied only one at a time, so don't expect to make large-scale changes quickly. The reports are decent but don't provide the level of detail we found in its rivals.

EPC does offer a useful comparison feature for change notification. Once a target computer was configured, we saved a baseline snapshot of it locally. We could then scan the target at a later time and view any changes from the baseline. Likewise, a template system could be used to compare other systems; this would be a useful capability in a centrally managed desktop/server environment.

eTrust Policy Compliance 7.4, Computer Associates International, (800) 225-5224. www.ca.comMike Fratto is a senior technology editor based in Network Computing's Syracuse University Real-World Labs®; he covers all security-related topics. Prior to joining this magazine, Mike worked as an independent consultant in central New York. Write to him at [email protected].

Post a comment or question on this story.

To test policy monitoring applications, we used a mix of our production and test machines in a live environment. We installed agents, where applicable, remotely from a central point or through start-up scripts. We used two Microsoft Windows 2000 Active Directory Forests, three Windows NT 4.0 Domains, several Windows standalone production servers, five Windows XP desktops and more than 50 Windows 2000 Professional desktops. We also monitored our production Sun Solaris 2.7 and 2.8 servers, and Red Hat 7.3 and 8.0 computers. In all, we had more than 100 computers in the test. Most interesting was the sheer disarray of our patch status and configuration. While we don't exactly run an ultra-secure shop, we found, using all the products, that several key servers were woefully out of date--a problem we rectified.

We used the predefined policies that came with each product to run initial discoveries and reports. That gave us time to learn each application. Once we got comfortable, we tailored reports to our needs. We also created an informal policy based on ISO 17799 statements and modeled that policy within each product. As we patched systems and made configuration changes, we monitored systems in each application to see how changes were reflected in each product.

R E V I E WSecurity Policy Monitors

Sorry,
your browser
is not Java
enabled

Welcome to

NETWORK COMPUTING's Interactive Report Card, v2. To launch it, click on the Interactive Report Card ® iconabove. The program components take a few moments to load.

Once launched, enter your own product feature weights and click the Recalc button. The Interactive Report Card ® will re-sort (and re-grade!) the products based on the new category weights you entered.

Click here for more information about our Interactive Report Card ®.