Report: Linux Vulnerabilities More Numerous And Severe Than Windows

Red Hat Enterprise Linux ES 3 has more high-severity risks than Windows Server 2003, and users are exposed to them for a longer period, according to a report released Tuesday.

A draft of the report was released last month and quickly attracted controversy for its methodology as well as allegations of ties between Microsoft and its researchers.

The full report confirms that Microsoft funded the study, and is sure to prompt further accusations of bias. But the researchers are providing the full methodology and challenging other security experts to test the legitimacy of their results.

Richard Ford, a research professor in the computer sciences department at the Florida Institute of Technology's College of Engineering, and Herbert Thompson, director of research and training at Security Innovation, a security technology provider, conducted the study. They used the ICAT Metabase, a database of vulnerabilities from the National Institute of Standards and Technology to measure the severity of the various vulnerabilities identified over the course of 2004. The report also tabulated the "days of risk" from the time vulnerabilities were publicly identified to the time they were fixed.

The report drew criticism from Red Hat. The head of the company's Security Response Team, Mark Cox, said on his blog,"Red Hat was not given an opportunity to examine the 'Role Comparison Report' or its data in advance of publication and we believe there to be inaccuracies in the published 'days of risk' metrics. These metrics are significantly different from our own findings based on data sets made publicly available by our Security Response Team."Researchers analyzed the two systems configured as Web servers with add-on software.

The findings:

- Researchers found that the Red Hat Linux had 3,893 total days of risk for all the risks classified as high severity, compared to 1,145 for Windows Server 2003.

- The average days of risk per vulnerability were 71.4 for Red Hat Enterprise Linux, compared with 31.3 for Windows Server.

- The team also looked at the vulnerability of the two systems to a port scan. They found that Red Hat Enterprise Linux had 77 high-severity vulnerabilities in its default configuration compared to 33 for Windows Server 2003, out of a total vulnerability count of 174 for Red Hat vs. 52 for Windows.However, Thompson admitted that the relative severity of a vulnerability doesn't necessarily correlate with how much damage an attack can cause. "I have seen multiple instances where l5 low severity vulnerabilities have been combined into an attack that would have done damage as bad as a high severity attack," he said. He also cautioned that the "attack surface" of both systems could be mitigated simply by turning on the firewalls that come with both Windows Server and Enterprise Linux.

In addition, Thompson admitted that the vulnerability counts lumped together the vulnerabilities found in Linux, as well as add-on open source software for the Apache web server, PHP scripting platform, and MySQL database. The report mentioned, though, that MySQL had five vulnerabilities that took more than 90 days to fix.

One critic of the report said it's difficult to measure the relative severity of vulnerabilities.

"There are so many ways to rate vulnerabilities and severities," said Johannes Ullrich, chief technology officer of the SANS Internet Storm Center, a service that reports on security vulnerabilities. "It's hard to come to up with an objective measure."

He also noted that a complete Linux distribution comes with a greater variety of software than Windows, making it larger, more complex, and more prone to vulnerabilities.

And the skills of the person running the system is extremely important to measuring how secure that system is, Ullrich added, "No operating system is secure unless you know how to apply the patches, configure the passwords, and disable services you don't need. You can't rely on a single security measure. You have to use firewalls and such to build up layered defenses. If you don't do that right, any operating system is vulnerable," he said.Thompson expects he and his co-researcher will face charges of bias on behalf of Microsoft due to the company's funding of the study. "One of the big issues was to get the methodology out there. We knew people would question the results because of Microsoft's involvement in funding," he said.

He and Ford submitted their research proposal to Microsoft, Microsoft evaluated the proposal, and decided to fund it. Thompson said the researchers also sent the methodology to various analysts, including Charles Kolodgy of IDC, and had it vetted by various academics as well as people at the RSA Conference. ."

Asked if the study would have been published if the results had come out in favor of Linux, Thompson responded, "They certainly gave us input but I'm sure the results would ultimately have been published no matter what the outcome was."

In the report, the researchers cited an earlier study by Forrester Research that also attracted a fair amount of criticism from Linux proponents. Thompson expects to hear reaction from them again. "I'm sure we'll get a fair amount of creative input based on who funded this study," he said. He pointed out, however, that Security Innovation has a wide range of clients, including Hewlett-Packard, Cisco, and IBM, and his aim was to encourage feedback from the technology community about how the methodology can be optimized for future studies. "Certainly I hope that when the criticism comes, it comes on the methodology and our acts instead of loud commentary on who funded this particular study," he said.

While the current study examines Windows Server and Red Hat Enterprise Linux in Web server configurations, Thompson and Ford plan to conduct future comparisons of database server and workstation roles.0