Service Delivery Modeling Strategies

Remember when IT could go into a finance meeting, spew impressive-sounding CPU-cycle measurements and come away with a fat budget? They're on to us now. Smart business leaders understand that IT services can be quantified and expressed in terms that have meaning to those who don't know a baud from a banana. Track availability and response times, the percentage of failed product queries, mean time to execute a shopping-cart transaction during peak hours, maximum number of down terminals during a given week--then we'll talk funding.

These measurements are the substance of SLAs (service-level agreements), to which IT can manage and the business can assign value. Service-management products bridge this synapse, translating raw performance numbers into business metrics. These products go by many names--business-service management, IT-service management, service-level management, service-level reporting, end-to-end service management, IT service-portfolio management ... the list goes on. To help readers match their companies' management processes and needs to product offerings, and determine how to strategically plan for and execute service management, we developed a questionnaire that would let us build an overview of vendors' often-overlapping and conflicting strategies. We asked nine questions, broken down along the lines of IT services mapping to business services, detailed infrastructure performance management, viability and SLA reporting.

We sent our survey to 14 service-management vendors; BMC, CA, EMC, Hewlett-Packard, Integrien, Managed Objects, Mercury, Oblicore, OpTier, ProactiveNet and Proxima Technology responded. Their product sets represent a comprehensive range of features, functions, markets and technology. IBM declined to participate without giving a reason, as did NewScale. Digital Fuel did not respond to our invitation. We also talked off-the-record with reference customers using service-management products.Hot Stuff

What's the latest buzz in service management? In an acronym, ITIL. All our vendors invoked the IT best practices of the IT Infrastructure Library. Not that ITIL is universally used. Shops we spoke with ranged from having ITIL certifications to instituting ITIL best practices, from having ITIL road maps to being totally oblivious to it. The continuum tracks to size: Large IT groups tended to have a centralized store of all management data, such as a CMDB (configuration management database), and ITIL awareness. Smaller shops just want to keep the infrastructure reliable and predictable, whatever the process is called. Our take is that ITIL is a factor for improved IT operations, but it's not driving service management except in vendors' PowerPoints (for an education on ITIL see the links listed at right).

Moreover, while Forrester Research predicts the SLM market will exceed a billion dollars by the end of 2007, big sales don't mean product sets are mature and stable. If you expect to write a check, download a dashboard and start managing services, you'll be disappointed. There are many choices, and finding where to start can be overwhelming. In this article we examine overall service-monitoring architectures. See the chart above left for our overview of vendor approaches to SLM, and you can read vendors' completed questionnaires by clicking on the links below.

Download Vendors Completed Questionnaires:
• BMC Survey Results
• CA Survey Results
• EMC Survey Results

• HP Survey Results
• Integrien Survey Results
• Managed Objects Survey Results
• Mercury Survey Results
• Oblicore Survey Results
• Optier Survey Results

• ProactiveNet Survey Results
• Proxima Technology Survey Results
Pick Your Approach

There's no magic tool that will align IT and business. SLM products surely try, though. The vendors we surveyed have two basic approaches: In a nutshell, respondents came at the problem from the bottom up, managing the infrastructure based on SLAs, or from the top down, knitting together everything that has anything to do with delivering IT services. We were impressed with Managed Objects' extremely wide top-down performance- and incident-gathering offerings that use existing network- and systems-management applications, agents, data stores and service desks. Bottom-up vendor Integrien is also adept at taking data from existing agents.

It's our experience that bottom-up products are easier to get running and more quickly provide actionable data than their top-down brethren. If yours is a very large shop, however, you probably have reliable performance data collected by different IT groups--no need to reinvent the data-collection wheel. The bigger you are, the more likely you'll need a top-down approach to tie together the myriad tools already configured and collecting IT infrastructure and service-desk metrics. But even in large, complex environments, the bottom-up approach can add value: It's easier to administer, while improving root-cause determination and the service delivery of the IT infrastructure.

Take It From the Top

The infrastructure-performance-management approach can better determine the root cause of a problem by statistically comparing collected data to historical norms. The top-down guys cast a wide net to include infrastructure performance and service-desk processes and tend to use the BSM (business-service management) moniker. Analyst firms such as Gartner and Forrester, as well as service-management vendors, further subcategorize BSM into functions including SLA reporting, application-dependency mapping, service monitoring, service management and service catalogs.This separation makes sense only from a marketecture point of view; for IT, these are related processes. The vendors we spoke with are busily adding, integrating and purchasing these functions so that, as their suites mature, service-monitoring options will likely be integrated. Vendors responding to our survey in this big top-down group include BMC, CA, EMC, HP, Managed Objects, Mercury, Oblicore and Proxima Technology. Integration with network- and systems-management apps is especially important for top-down SLM vendors. However, there is a limit--BMC, CA, EMC and HP, which have their own network- and systems-management products, integrate service management more deeply with proprietary sources. Oblicore, Proxima Technologies, Mercury and Managed Objects are more flexible and reliant on incumbent network- and systems-management products, therefore providing wider integration.

All the products from vendors in the top-down group gather performance and service stats from many sources. Products from Managed Objects, Oblicore and Proxima Technology collect data from a variety of IT performance and service sources. These vendors don't offer separate performance-monitoring products. The offerings from BMC, CA, EMC, HP and Mercury gather disparate data as well, but also have proprietary performance and service desks. This gives the latter group a superset of functionality.

It's ideal to collect data with the same product that is managing services because, in general, integration will be tighter. However, if you have point products in place collecting data, the investment won't be lost. All these top-down vendors group infrastructure in accordance with the application or service it delivers.Not surprisingly, archetypical BSM vendor Managed Objects focuses its product on creating an asset listing of hardware, software and applications, and the services they enable. Because Managed Objects' tool isn't polling or deploying agents on systems, the company rattled off the most impressive list of out-of-the-box integrations, including working with products from BMC, CA, EMC, HP and IBM.

We were a little surprised with EMC's response, in the context of its recent purchase of the SMARTS network and application fault-management products. SMARTS is best known for its patented Codebook Correlation Technology, which automatically determines root cause based on predetermined symptoms, but not service management for applications. SMARTS has now added application event correlation to its functionality, throwing its hat into the service-management ring by creating collections of IT infrastructure representing IT services.

EMC-SMARTS is not alone in this claim; all our top-down vendors say they capture and normalize data into some sort of centralized asset data store, which they correctly label a CMDB. CA has been making this point with its Jasmine database for years. One thing is for sure: You'll need to gain expertise in whichever database/CMDB you entrust with your service-management eggs. ITIL's definition of a CMDB specifies only a minimal schema. Each vendor supports multiple database platforms and the minimum schema to be a CMDB; they will also extend the stored data and, more than likely, store it in more than a single table and database.

Let IT Flow

Top-down BSM vendors go beyond IT infrastructure performance, integrating service-desk IT workflow and problem management into their view of how well IT is delivering services. This is not something that infrastructure-management vendors OpTier and ProactiveNet include. The bottom-up offerings do feature integration to open trouble tickets based on threshold violations, but don't consider incidents when determining SLA adherence. This is a shortfall in these products compared with their top-down brethren. How big a shortfall? That depends on where your performance failures come from.If IT infrastructure failures represent the majority of what you care about, the bottom-up approach of leveraging statistical anomalies promises to predict and avoid failures--rendering service-desk complaints moot. In contrast, the more inclusive top-down collection of infrastructure and service-desk incidents can cast a net around non-IT issues. As an example, HP provided a case study that demonstrated how process management could be integrated into service monitoring. A cellular phone company recognized "abnormal sales" when, at 11 a.m. on a Saturday, sales volume was below normal. The cause: issues with manual activation at a retail store. The monitoring suite alerted helpdesk personnel, who were able to give the store a call and offer guidance.

BMC's product integrates with its own service desk, Remedy, as CA's does with its Unicenter Service Desk. HP's and Managed Objects' offerings are a bit more open, working with HP OpenView, Service Desk and Remedy. In addition, HP recently purchased Peregrine; we expect to see future integration here. Mercury cited its self-service portal as service integration. EMC's product has the widest included integration, claiming to work with HP OpenView, Service Desk, Peregrine, CA Unicenter Service Desk and Remedy. Even bottom-up vendor Integrien has its own Alive service desk.

Oblicore didn't claim any external helpdesk integration, but one of its large service-provider customers we talked with described a homegrown process it had implemented using incident data. And, as an interesting aside, the customer uses only incident data to create SLA reports with Oblicore. It felt that real-time monitoring, while good for IT Infrastructure diagnostics and capacity planning, is too unreliable as a tracking method for SLA compliance. Whether this stance makes sense depends on the thoroughness of your monitoring instrumentation. In this case, the provider had to monitor handset availability, which it was unable to do reliably.Weather Central

In addition to service-desk and process monitoring, we asked vendors about their ability to monitor datacenter, environmental, network, systems, application and end-user perspectives.

End-to-end monitoring of end-user transactions is one way to detect a failure without having to track all intervening components. Again, this is a problem of instrumenting enough locations to plug holes. Most of the products come with agents that can send robotic transactions that simulate real user transactions and offer a useful--but limited--view of the user experience. BMC, CA, HP and OpTier from our survey, and user-monitoring specialists Coradiant, Compuware and Quest Software, are able to monitor actual transactions at aggregation points, for example, in front of a Web server.

End-user monitoring is powerful in two important ways. First, it's a customer view; and second, it can see the salient business action--the transaction. End-user monitoring does nothing to diagnose a router, firewall or server problem, but without it, correlation between devices and apps on the service chain delivering transactions to the customer must be perfect--an unlikely occurrence.

Network, systems and application monitoring was, not surprisingly, supported by all the vendors. But we were glad to see that support for the unsexy monitoring of datacenter environmentals was strong as well: CA, EMC, HP, Mercury, Oblicore and Proxima Technologies all have third-party integrations to track factors like humidity, temperature and power.Another common assertion, regardless of the approach--up or down--are business-service-oriented dashboards. Each vendor lays claim to this simple GUI overview as if its were unique but, in reality, they all have gauges and portal interfaces for simple business views, and more complex IT infrastructure drill-downs. All the vendors show real-time or near real-time data with SLA values and current service status.

Many IT pros consider these dashboards little more than eye candy. We recently asked a high-level exec at a large management vendor about the validity of that view. He told us that most GUI interfaces are, indeed, mainly window dressing intended to sell product to C-level executives. He added that many business-side execs rely on dashboards as the ultimate correlated and communicated view of IT services to and for the business. It's unsettling that a flashy display has that much sway, but it apparently does, so getting a customizable, integrated one is important.

Bottoms Up

Coming at service management from the bottom up are Integrien, ProactiveNet and OpTier. All monitor baselines and automatically set thresholds for network, systems and applications.

On face value, this seems like a subset of the infrastructure monitoring done and consumed by BSM vendors. But in reality, the out-of-the-box automation provided by these infrastructure-management vendors-- which admittedly isn't as broad in scope as that offered by BSM vendors and doesn't including service-desk incidents or IT workflow--is more focused on reducing MTTR (Mean Time to Recovery) and MTBF (Mean Time Between Failures) and costs less to implement and own.All the products in the bottom-up group collect and analyze data in real time, and record and analyze performance over time. They capture data using proprietary agents and generic collection techniques, such as SNMP, JMX (Java Management Extensions), WMI (Windows Management Instrumentation), database queries and log files.

By recording the historical performance load of networks, systems and applications, this class of product creates baselines that are unique to a particular service at a particular time. Traffic, CPU and transaction load on the second Tuesday at noon are separate from the third Tuesday at noon. This performance data combines with min/max/average and 90th percentile trends to create a snapshot of "normal" processing. These snapshots provide automated thresholds against which these products can alert.

Integrien claims to kick this process up a notch or two by correlating the events leading up to a threshold violation as predictors of the violation. So if, prior to a longer-than-normal Web transaction, CPU load and number of users increase by X percent, the next time both those metrics for that service group see an X percent increase, a prediction regarding the threshold violation will be made with a particular confidence level. The math, metrics tracked and number of snapshots are more complicated than outlined here, but this the basic idea.

OpTier adds a different twist, using dynamic thresholds and transaction monitoring to prioritize processing dynamically through queue management. Indeed, transactions are the basic unit of measurement for OpTier, which monitors and prioritizes them across tiers. Transaction categories include user, group and origin. Performance of the underlying IT infrastructure is mapped to these partitions, providing a unique tracking of IT infrastructure resource utilization. Say a shopping cart purchase arrives for processing at the same time as a store locator search. The shopping cart will be processed while the search is queued. OpTier claims out-of-the-box integration with database, EAI and application servers aimed at supporting SOA.

Service FeaturesManaged Objects pointed to a service-management gap, which we agree is exactly the problem. The gap lies between the services delivered by IT and monitoring of the infrastructure. All the vendors stressed that infrastructure performance must be tied to deliverables the business side understands.

But there's another, higher, level of service translation that relates IT infrastructure performance to real business goals, such as revenue or customer satisfaction. The vendor response when we asked how to fill that gap was mixed, but similar. BMC and EMC candidly admit that integration with external accounting systems is required to make higher-level connections. CA relies on its own products and resources to create these links but still requires some integration. ProactiveNet requires that a revenue value be defined for all watched elements, and OpTier uses its transaction monitoring to identify important transactions. Others were vague on this point; Integrien was the only vendor to flatly state that it doesn't map to business metrics, like revenue.

Modeling service changes is another valuable function, displaying what services could be impacted by a change to the infrastructure. These modelings are not like those from Opnet Technologies or Analytical Engines, where the actual network and systems infrastructure is modeled for load, nor are they like offerings from AlterPoint, Opsware or Voyence, where network configuration products determine possible impacts based on changes to configuration file syntax and policy violations. Rather, this is a modeling that looks at a broader view of the devices that are going to be changed and what services they support. For example, as infrastructure changes are proposed, associated SLAs can be checked for maintenance windows and uptime requirements. In the case of redundant infrastructure, the lack of SLA impact can be flagged.

All the top-down vendors, with the exception of Proxima Technology, support this function. ProactiveNet was on the fence about support, not claiming to model data but able to project the likelihood of SLA compliance based on historical data. OpTier relies on third-party integration with IT service-optimization vendor TeamQuest to model changes and likely service impacts. This is a better, deeper and truer modeling of resources as compared with just identifying impacted services.

Integrien and ProactiveNet are the only vendors that didn't cite integration with an automated application dependency-mapping function--in contrast, Managed Objects integrates with more than five unique vendors. Both Integrien and ProactiveNet plan to add this functionality, but stressed that they don't need it for root-cause functionality, as the top-down service management vendors do.Because both Integrien and ProactiveNet create continuous performance snapshots, which they statistically compare and use for automated root-cause and threshold calculations, they told us that dependency mapping holds less value for them. So why are both vendors considering adding this function? Because it will automate the gathering of network, systems and applications into groups, automatically placing all of the infrastructure involved in providing a particular service together; this currently requires manual configuration.

Trusted Partners

We asked for some viability assurance in terms of units sold or number of customers, but responsiveness was mixed. The big guys had lots to say, while the small fry were a bit shy.

BMC and HP disclosed revenues from federal 10q fillings, listing earnings by business line for the last eight quarters. BMC's most recent quarter came in at over $360 million, slightly higher than it has been but very consistent overall. HP was a little over $304 million for the last quarter, a year-over-year increase of 29 percent.

If number of customers make a vendor reliable, CA claims more than 5,000, Mercury tens of thousands, Proxima Technologies 150 and ProactiveNet 170. But none of these vendors shared revenues for recent quarters. On the smaller side, Oblicore claims more than 45 customers, Integrien cites a dozen after just a year, and OpTier has sold to 10 customers and claims to have another 20 in process. Again, no revenue sharing.We asked the vendors for both high and low pricing for service-management suites, as well as average cost. This, too, was a mixed bag--vendors were cagey, and we expected as much. The information we gathered is presented in the chart below; hopefully you can get a ballpark feel for purchase price. In our experience, service management starts at around $50,000 and goes up depending on the amount of monitoring done. A 10,000-node network could easily cost $250,000.

Road Map

We asked vendors where they see service management headed. CA says it plans to make its products easier to use by adding additional wizards; it also wants to integrate change and configuration management through out-of-the-box integration with its helpdesk suite.

EMC is busy integrating SMARTS with its storage and virtualization lines. Another recent EMC purchase, of nLayers (which provides a unique CMDB that automatically tracks application interdependencies), gives the SMARTS application groups the ability to extend already strong Layer 2 and Layer 3 root-cause analyses up to Layer 7.

Integrien aims to continue adding intelligent alerting with recommended root-cause actions, believing that will increase customers' comfort with its diagnoses. Integrien adds that partnerships with other, as yet unnamed, vendors will then allow for automated configuration changes based on its expert root-cause analysis.Managed Objects says it plans on automating to support infrastructure that is provisioned on demand and will strive to be more accurate directly out of the box, a refreshing admission. The company is also working on support for any eventual SOA standard, but didn't declare a preference.

OpTier is looking to add more out-of-the-box integration with network devices and transaction environments, though it didn't provide specifics. It also mentioned additional integration with change-management, configuration-management and service-modeling vendors though, again, no specifics were given.

Finally, Proxima Technology is adding additional grouping functionality, point-and-click dashboard features and a Web services integration adapter for collecting data in a Web services environment.

HP, Mercury and Oblicore weren't forthcoming with plans for the future.

Bruce Boardman, executive editor of Network Computing, tests and writes about network and systems management. He has 12 years' experience managing networks and distributed computing for a financial service provider. Write to him at [email protected].Executive SummaryBe it SLM, BSM or your acronym of choice, IT-business alignment sounds great in theory, but is devilishly difficult in practice. It must happen, however, because, technology has lost that miasma of mystery, that veil of vagueness. The sorcery factor is gone, and with it, the free flow of dollars. Accountability is all the rage. In fact, Gartner estimates that by 2008, 80 percent of Global 1000 organizations will hire an IT finance manager or comparable individual charged with optimizing the finances of the IT organization. Shudder.

To help cut through the service-management FUD, we developed a list of questions designed to determine 11 vendors' strategies in key areas, including mapping IT services to business services, infrastructure performance management, viability and service-level agreement reporting. Here, we call out highlights of the responses and discuss cost, future plans and the two major approaches to service management (see features chart for detailing suite capabilities). You can find the vendors' full responses online on the first page of this article.Questions:

We asked vendors for the following information to get an overview of strategies in key areas, including mapping IT services to business services, infrastructure performance management, viability and SLA reporting.

1. At least two reference customer contacts for interview.

Interviews will take place between NWC and the customer without the vendor. The interview will be used to understand Service Management issues, challenges and strategies. We can shield the contact's identity by saying, for example, "an IT manager for a 10,000 employee manufacturing firm."2. Service-management-features architecture regarding

» Automation

» Integration

» Processing

3. An annotated architectural diagram in jpeg or bitmap format4. Service-management metrics reported/measured by product, categorized by the following:

» Technical IT infrastructure, i.e. CPU, IO, MTTR

» Financial costs impact of service

5. Commercial by units sold, delivered, customers serviced, etc.

6.Product pricing:» High and low ranges

» Average initial purchase

7. Project implementation planning guidelines

8. Service-management product road maps--including market segmentation and underlying assumptions, like required customer process maturity--for the next two years

» Benefits of your service-management solutions to IT and to the business» Organizational profiles of businesses that will best be able to take advantage of your service-management offering(s). Include factors such as size of business by gross revenue, market, industry, technology deployed, and processes implemented.

9. Your company's market presence, including the following:

» Number of customers

» Service-management revenue for fiscal year 2005

» Recent acquisitions, partnerships and technology developments related to service management» Third-party indicators of market leadership (for instance, analyst rankings or awards)

The above information must be provided to be included in the service management analysis review. If your company has additional material in the way of research, white papers and/or demonstrations that you feel would be helpful, please include it. If your have questions, contact Bruce Boardman at [email protected].