Does Legato Outpace NSI?

Legato Systems Inc. (Nasdaq: LGTO) still has the urge to strut its stuff, even as it's preparing to merge with EMC Corp. (NYSE: EMC) -- Legato trumpeted a test today showing that its replication software is faster than competitor NSI Softwares (see Legato: We're Faster Than NSI).

Legato announced that VeriTest, Lionbridge Technologies Inc.’s testing division, has conducted a study showing that its RepliStor software replicates writes nearly five times faster than NSI’s Double-Take software.

“I think it’s a good proof point,” says Legato CTO George Symons. “It shows that we can really scale to meet the needs of customers… from small to medium sized businesses all the way up to the departmental level.”

VeriTest conducted the tests over the summer, and completed the “independent” study, which was both commissioned by Legato and based on a Legato load-generation tool, last month.

While the study does indeed indicate that RepliStor is faster than Double-Take in nearly every instance, Don Beeler, NSI's chairman, president and CEO, insists that they are completely invalid. “The one thing they didn’t do was an end-to-end test,” he says, pointing out that Legato not only paid for the tests, but also provided the loads used. “The data set was completely contrived... And just measuring performance between components isn’t really interesting.”Steve Kenniston, an Enterprise Storage Group Inc. analyst, agrees that only complete end-to-end tests are really of value. “On the testing side, hardly anyone in IT believes these reports, because they don't test real-world application data sets or test with real-world configurations to make them viable,” he says. “A lot of times companies do these types of tests against strong competitors just to create FUD.”

While Beeler claims that NSI would certainly prevail in a real-world scenario, he says the company isn’t planning any retaliation tests. Customers, he says, are much more interested in things like how stable the product is, and how much CPU load they can expect from it.

“Legato should spend more time building more stable products than manufacturing test results,” he scoffs.

Regardless, though, Beeler believes that Legato's performance comparison test is in fact good publicity for his company. “When you’re in the No. 1 spot, people are continuously throwing rocks,” he says. “It’s very nice of them to recognize us as an industry leader. This is just one more validation of that.”

VeriTest says there was nothing manufactured about its test results. “VeriTest believes that integrity and disclosure benefit everyone involved in the testing process, and we're committed to these guiding values,” a company spokeswoman writes in an email.Legato's Symons, for his part, insists that the results are legit. He also says that Legato commissioned the test not to bash NSI, but more out of scientific curiosity. “We kept hearing back from customers that we were faster, but we didn’t know what that meant,” he says. “We wanted to quantify that in a way that made sense.”

So how does the study quantify the performance differences? The tests consisted of both manual and real-time replication of data from a source to a target computer system. VeriTest conducted each of the automated replication tests at four different connection speeds: T1, 10 Mbit/s, 100 Mbit/s, and 1 Gbit/s, and found, among other things, that Legato’s software synchronously replicates data 4.92 times faster than NSI's Double-Take over T1 wide-area networks, and is up to four times faster over a 10-Mbit/s local-area network connection.

Figure 1: Source: VeriTest

For the manual tests, VeriTest executed a full synchronization of the data from the source system to the target system, again at the four different connection speeds. The test division also carried out four incremental synchronizations, requiring writes of 0 percent, 5 percent, 25 percent, and 100 percent changed data. These tests revealed that RepliStor was 25 percent faster than Double-Take at 1 Gbit/s during the initial, full synchronization. As for the incremental synchronizations, the test showed that Legato’s software was a whopping 188 percent faster than Double-Take when 0 percent of the data had changed, and 12 percent faster after 100 percent of the data had changed.

Figure 2: Elapsed Times for Manual Replication in SecondsSource: VeriTest

RepliStor didn’t win at every level, however. The VeriTest study shows, for instance, that at 1000 and 100 Mbit/s, NSI’s software was 27.9 percent faster on the incremental synchronizations at 5 percent data change and 8.4 percent faster at 25 percent data change.

All in all, however, Legato came out of the study looking like a true Rocky in the ring -- the opponent beaten to a pulp.

No matter how accurate the test data may be, the huge performance difference seems a bit strange, especially in light of the patent infringement lawsuit Legato filed against NSI earlier this year (see Legato Lobs Lawsuit at NSI). The company has charged that the smaller vendor in fact swiped its RepliStor patents to create Double-Take.

“There are some other things we can do in the product that are different than the patented ideas that are in there,” Symons says to explain how two products supposedly based on the same patent show such deviating performance.

Legato also announced today that VeriTest has certified RepliStor for Windows 2000 and Windows Server 2003.— Eugénie Larson, Senior Editor, Byte and Switch