Survivor's Guide to 2007: Network and Systems Management

Even Among IT Pros, a skeptical bunch, NSM practitioners stand out. Years of vendor hype followed by dashed expectations have made us cynical. In this year's NWC Reader Poll, seven of every 10 respondents opined that no one has ever finished a network management platform deployment. That's got to be a slight exaggeration.

Our outlook should improve as significant transformations shake up NSM on both the technology and human fronts. On the tech side, the moves are driven in large part by higher CMDB (configuration management database) usage that will enable cross-domain analytics and integrated and consistent approaches to infrastructure and application discovery. Kiss those siloed, toolset-by-toolset affairs goodbye. It's a matter of survival: If you're not worried about new technologies that demand a more unified, integrated approach to management, most notably SOAs (service-oriented architectures), you're not paying attention.On a human level, the move toward addressing network, systems and application management as a continuum, supportive of process requirements and best practices, is putting subtle but very real pressure on IT organizational structures. Old ways of doing business--and the work identities of more than a few IT professionals--are under fire.

The past was about managing discrete systems or devices, using niche tools wielded by closet dwellers with specialized expertise. The future will demand the same, if not more, intelligence, plus significant communication between IT and business colleagues--in plain English, please.

What does that mean for tech pros? Managing individual devices has become more challenging. Networks are denser, more complex and more frequently subject to change. Managing virtualized environments and supporting huge application payloads--some n-tiered, Web-based or otherwise modular--are also raising the pressure level.

All this is driving a trend toward hiring well-rounded professionals whose deep device and engineering expertise is combined with a collaborative mind-set. Working as a "genius in isolation" with an array of scripts and processes that can never be shared won't get the job done. We must work as a team, using standardized methodologies and consistent approaches for uniting IT across domains.Of course, a unified approach with standardized processes was always a good idea. After all, ITIL (IT Infrastructure Library) goes back to the 1980s, and more and more management technologies support these requirements. Although no single NSM tool can provide all the capabilities you need, hundreds of management vendors are trying to design for this collaborative, process-centric market. The enterprise systems-management market is slowly becoming civilized.

Movers And Shakers

The transformation won't be complete this year, or even next. But we'll get a good start thanks to a handful of technologies--some emerging, some old friends--that will become more prominent.

>> DISCOVERY: The hottest area has been application dependency mapping--the discovery of application-to-infrastructure interdependencies--which would seem defined more by company acquisitions than conventional metrics. But core requirements to provide accurate and consistent views of discovered "realities" will remain an area to watch. These views include everything from Layer 2 and Layer 3 topologies through Layer 7 and into application components, content and dynamics.

Why is discovery important? Because the IT infrastructure and its applications are becoming more unpredictable all the time. And while SOAs aren't yet keeping most NOC professionals up at night, trust us on this: They will herald a shrill and monstrous awakening to the need for accurate and dynamic visibility within the next five years.>> CONFIGURATION MANAGEMENT: It shouldn't be surprising, given the importance of the CMDB, that advanced configuration management capabilities are crucial to the transformation of IT. Companies such as BladeLogic and Configuresoft on the systems side; AlterPoint, Intelliden and Voyence on the network side; and Opsware on both fronts, represent critical areas to watch and invest in.

And not surprisingly, configuration management, like the CMDB itself, is more like an octopus than a clam, reaching into many areas, including performance and availability, security and compliance, and, of course, change and IT governance.

>> CROSS-DOMAIN ANALYTICS: If you think about what a CMDB enables--a CMDB is, indeed, nothing more than an enabler--the next logical leap is that it allows for superior analytics. After all, CMDBs are about nothing more or less than creating a level playing field on which advanced real-time and non-real-time analytic capabilities can perform like Olympic athletes.

This is true whether the analytics involve correlation or signature-based comparators, predictive algorithms or data mining, or market-disruptive technologies such as Chaos Theory and neural networking. We predict within five years, the focus on data integration and reconciliation from CMDB adoption will lead to an acute interest in analytics. Vendors such as Integrien, Netuitive and ProactiveNet are out in the marketing wilderness today. They won't seem quite so alone in a few years.

>> APPLICATION FLOW MANAGEMENT: Take this as an example of a broader concern--instrumentation and the need for true real-time awareness of service performance across the infrastructure. We define application-flow management as including both packet analysis and conventional pairs-based traffic-volume monitoring. It can support everything from performance management, capacity planning and security management via anomaly detection, to chargeback and service accounting and route analytics. In fact, it represents the most singularly interesting network management area not yet co-opted by framework or platform vendors.And as SOAs become more pervasive, application-flow monitoring and active management will become more important. Key players here include Apparent Networks, Cisco Systems, Crannog Software, Fluke Networks, NetQoS, NetScout Systems, Network General, and Network Physics and Packet Design for route analytics.

>> OUT-OF-BAND: You weren't expecting this, right? But you should be. Out-of-band technologies--the big kahunas are Avocent and Raritan--will become a core part of the requirement to ensure that remote environments can be managed effectively, even when in-band network disruptions occur and systems are out of commission.

Put succinctly, the enterprise NSM market has focused on monitoring far more than fixing. But as the landscape evolves, the need for intelligent surgeons' hands and effective diagnosticians will become more apparent. The future will require viewing OOB as an integrated foundational technology rather than a separate niche market.

Into Its Own

One core transformer we've watched for several years is CMDB. OK, it's not really a database, and it's not limited to configuration management in the vernacular meaning. Rather, the CMDB is an ITIL offshoot, and in ITIL, "configuration management" denotes a documentation of relationships ranging from topologies and configuration information, to customer and service relationships, to service and infrastructure interdependencies. The CMDB also extends to cost- and contractual-related information, and to SLAs via performance and availability data.Attaining this cosmic view of both desired and discovered states requires more political will than technical insight: Removing the clutter of toolset redundancy, enforcing sharing and assigning responsibilities to maintain sources of record are the greatest CMDB challenges.

Of course, this will take time. Lacking a phased approach to building a more consistent, collaborative environment, the CMDB could, in the words of one skeptic, turn out to be "a Utopian vision that will collapse of its own weight."

Maybe. But for now, CMDB adoption is steadily on the rise, despite ongoing confusion (see chart above). Most IT groups are still wrestling with what philosophers would call ontological questions--metaphysical musings about the definition and direction of the CMDB itself. What is a CMDB? What should it be? And why is it probably the single most significant bellwether of the cataclysmic transformation of IT management?

Split Personality

Enterprise Management Associates found that interest in CMDB adoption has outstripped interest and awareness in ITIL. This is both disturbing, as once again process considerations have been shoved aside for technology, and heartening because it indicates that the age-old need to integrate and reconcile management investments is finally getting, qualitatively, a new life.

EMA also has documented two fundamentally different CMDB systems, each of which may house many subsystems and variants. A desired-state CMDB is designed to support review and approval through a Change Advisory Board (ITIL's term). A discovered-state CMDB is targeted at real-time or near-real-time requirements for managing service performance across multiple data sources.

Typically, the desired-state CMDB is a database. A discovered-state CMDB is more likely a means of policy-based, time-reconciled visual access to multiple data sources. The two systems are complementary and can be used so that discovered state, based on policy, can be applied to populate the desired state through, in some cases, a system of caching.

We figure we're in Year 2 of what may well be a 10-to-20-year evolution process, so final exposition of what these two systems will look like remains "written in water."Platform For Change

A large part of the CMDB tidal wave is a redefinition of management platforms, including those from BMC, CA, HP and IBM as well as new entrants such as EMC. We see next-generation platforms being defined as vehicles for the intelligent integration of existing and planned management investments, meaning the platform of choice will necessarily become a war for the position of core, or desired-state, CMDB.

This is already happening: The battle for CMDB preeminence will separate winners from losers in platform choice. And this is an enlightened perspective. Rather than an imperialistic, end-to-end, invasion, platforms will become the ultimate vehicles to enable and enhance brand choice in addressing critical management requirements. For more on CMDB, see "So Much More Than Just a Pretty Database".

Be Not Afraid

If this NSM landscape seems like an alien planet, that's because for most shops, it is. But it's also one rich in new resources and opportunities.

From a product standpoint, the winning strategies will be those in which ease of deployment and adaptability are fundamental design criteria, not afterthoughts. If a vendor touts advanced analytics but requires months of assessment and tuning, let your sales reps know their offerings aren't so advanced after all.

Internally, keep in mind the close relationship among technology choices, process or best-practice initiatives, and organizational structures and patterns of communication. To optimize your investments in management products, think of them as vehicles for improving process and organizational dynamics. Ideally, these investments are never made in a vacuum. Tools, process and organization are inseparable--each is a catalyst impacting the others. Understanding this reality will minimize roadblocks while opening up exciting new opportunities. So Much More Than Just A Pretty Database

The CMDB is focused on integrating management solutions in support of better process efficiencies. While database technologies, and in particular those supporting the integration and reconciliation of multibrand investments, are central to CMDB planning, so are discovery, identity management, access control and workflow.

The CMDB is therefore not a "thing." It is a landscape in which trusted sources of information are identified and ownership is assigned so that consistent perspectives on configuration, topology, performance, assets and more can be shared effectively and without redundancy.

This first step is the most difficult, not because of technology issues but because of politics. We live in a universe of multiple discovery systems, multiple trouble-ticketing systems and niche tools that will be relinquished only after sometimes violent debate. But standardizing on consistent sources of information is a non-negotiable requirement for moving forward. To succeed requires committed investments in architecture and process expertise and the willingness to engage in ongoing, bidirectional dialogue across all IT silos--talking to each other in a way that's never before been required, or even invited. What's Driving The It Transformation?

If you've been paying attention, you've noticed over the past several years signs of a move to address network, system and application management as a continuum, supportive of process requirements and best practices. That vision puts subtle but very real pressure for change on IT organizational structures.After the technology bust at the turn of this century, the outcry for IT to show more accountability reached deafening levels. Although things are easing somewhat in terms of business confidence and willingness to make technology investments, the drive toward accountability hasn't--and shouldn't--go away. Rather than treating IT organizations as a quasi-academic mélange of professional skill sets grouped by knowledge, education and background, IT is evolving toward a more integrated set of competencies. What better way to support business value and quantify our contributions?

Think about it: Virtually all business structures depend on these axes--quality, cost and relevance--to navigate and assess their effectiveness. But this is still a very new set of principles for IT organizations--especially NSM groups--that have been protected for years by a brick wall of arcane acronyms and crusty attitudes that insulated us from the outside world.

The last major IT management change even approaching this magnitude was client/server and the rise of SNMP over SNA--a transformation many of us watched and took sides on, sometimes even marveled at. But at least client/server was about a visible system, a tangible structure you could get your arms around.

This new change, reflected in technologies like CMDB and SOAs on the one hand, and ITIL and BSM initiatives and a transformation of asset management on the other, is even more vast, far less tangible and, in the end, far less predictable. Rather than changing what IT must manage, it is changing, inalterably, how IT will go about the business of managing its services in support of needs that are business-defined, rather than purely technology-defined.

Chris Matney has 20 years technology and business development experience, with a focus on IT systems management. He currently leads Enterprise Management Associates' consulting practice, including its research into CMDB technology, and he created EMA's 8-Step IT Assessment Methodology. Write to him at [email protected]