Web Analytics Products

Let the Shopping Begin

We identified various SU Athletics customer segments and created funnels--predetermined pathways to desired results--to track users' behavior as they bought tickets or audio subscriptions through the site. We spent time

with the professional services or presales support groups from each vendor to identify the best way to do this and to learn about other sources of information we could tap using the vendor's product. We evaluated each service based on e-commerce components, flexibility and customization of reporting tools, ease of use, customer support, and price. We found that all the online services offer good performance (for info on gathering Internet performance metrics, check out our review of Gomez Performance Network 6.0).

In testing these services, we first wanted to make sure each would handle our basic reporting requirements--path and funnel analysis; dynamic segmentation; search engine/keyword optimization; basic visit and visitor reports, including length of time on site or pages; geo-metrics, which show geographical information on visitors; and browser analysis. We also evaluated each vendor's professional services options.We've been covering Web analytics for many years and are accustomed to the lengthy reports that can be provided on an on-demand or daily basis. Given that experience, we want to make one thing clear: You can't just buy a service, read some reports and expect to improve your company's Web business. Effective Web analytics requires that organizations constantly analyze reports and, more important, act on the information found in them. This is sometimes easier said than done. At a recent conference of Web administrators, we asked 200 participants how many use commercial Web analytics software. All but seven raised their hands. Then we asked how many had made tweaks to their sites in the past three months based on information found in the reports. No hands went up. Zero.

In addition, implementing an e-commerce Web analytic service isn't simple. Typically, e-commerce pages require customization (changes to page code) for the Web analytics software to generate useful information. Because of this, customer support accounted for 15 percent of our score. Specifically, we looked at what professional services each vendor offered--dedicated support personnel, training, Webinars/seminars and, of course, ongoing consultation. Of the vendors whose services we tested, only Sane Solutions, WebSideStory and WebTrends offer a full lineup of professional services, including help in analyzing reports and ongoing evaluation and recommendations. The other participating vendors provide seminars, training sessions and initial support, but not the level of professional services we feel is essential for enterprise Web sites.

We also evaluated content groups, campaign tracking and support for multilevel product-category information for the e-commerce and merchandising aspects of our review. Finally, we considered the offerings' ability to integrate information with third-party products, such as business intelligence, customer relationship management and sales force automation tools.

Analyze ThisOne of the key service differentiators is dynamic segmentation. Setting a customer segment lets you group some visitors based on selected criteria, then analyze their behavior. "New visitors" versus "returning visitors" is a common example, and most vendors create these two segments by default. Alternatively, you might want to create a segment based on visitors who subscribe to your newsletter. Some vendors let you create segments based on previously collected data on the fly. This is useful for sorting data to detect patterns and answer questions. All the vendors except Urchin, WebtraffIQ and WebTrends let you do this.

WebSideStory's implementation, called active segmentation, replaces its legacy "population groups" as the preferred method of viewing visitor segmentation information. The advantage to active segmentation is that it lets you combine visitor characteristics to create segments on the fly. In our tests, however, the results weren't immediate--after setting up a segment, we had to wait an hour for the status to change from "pending" to "live" while WebSideStory's server crunched the numbers. In addition, we were limited to 10 dynamic segregations per month. WebTrends has segmentation, but it's not what we'd call dynamic--we typically had to wait a day to see the data show up in our reports. Only ClickTracks, Fireclick and Sane Solutions allow unlimited slicing and dicing with real-time updates. Another potential problem is that active segments generally apply going forward--with WebSideStory, for example, the start date for a segment can't be set earlier than three months back from the current date, so historical analysis is limited.

The Players

If you have enough in-house chops so that professional services aren't vital to your organization, consider the services from Fireclick and ClickTracks. Both scored high for ease of use. Fireclick's Warehouse is extremely flexible and customizable, and we liked ClickTracks' philosophy that the only modification our page code needed was the insertion of JavaScript. Although we feel ClickTracks aims more at the small and midsize business market than the enterprise, its hosted service is incredibly well-designed and powerful for the price, which was the second lowest of the services we tested. Urchin was in the same boat: Its offering has some unique features and is easy to use, but it lacks professional services and its pricing model won't scale well for the enterprise.

Bringing up the rear was WebtraffIQ, whose lack of dynamic segregation and customization capabilities make it one of weakest services we tested. It is also one of the most expensive, at $60,000 per year.Our Editor's Choice award goes to WebSideStory's HBX On-Demand Web Analytics 2.5, which edged out Sane Solutions' NetTracker On-Demand Service by a tiny margin. Although NetTracker sports some unique capabilities, including the ability to set up roles, HBX On-Demand provides impressive flexibility in its e-commerce analytics and the professional services we feel are so important. Web analytics are useful only if an organization takes the necessary steps to modify its Web site based on information gleaned. Professional services go a long way to help make that happen.

Of all the participating vendors, WebSideStory appeared the most excited to work with us and offered the most support. Now we're not saying this is a sure sign of excellent customer service, any more than a vendor blowing off a reviewer is an indication that it will be less than attentive to users. But given a choice, we'll take a culture of responsiveness anytime.

The JavaScript code package for WebSideStory was the largest among the services we tested. It consists of an include file and a big set of customizable variables. Although the code size in itself is not a problem, the use of custom parameters for almost all tracking features gets cumbersome fast. For example, many of the pages on our test site shared the same name in the