New Research: IT Automation

IT is caught between requests for new applications, services, and device support and demands from management to keep budgets and staffing levels static. Operational costs consume most of IT's resources, our InformationWeek Analytics 2011 IT Automation Survey shows. Seventy-three percent of our 388 respondents allocate less than 25% of their budgets to new service development--a situation unlikely to change anytime soon, as 82% say their 2012 new-services development budgets will stay level or shrink.

Yet 57% see the volume of requests for new and enhanced services growing in 2012; just 5% expect a decrease.

To skew the equation in favor of greater business-enhancing innovation requires work in four areas: automation, a shift to an IT services portfolio, more rigorous governance using industry best practices with clear service and process definitions, and a reassessment of just what services it makes sense to deliver internally. Automation in particular can help reduce operational costs and improve efficiency while minimizing human error by turning manual, ad hoc administrative tasks into repeatable, systematic processes. Automated processes also leave an electronic audit trail that can be used to diagnose problems and document policy compliance and service quality.

Still, the concept of automation has always created some cognitive dissonance. We like it in concept, but our actions tell another story. In our survey, an overwhelming 86% say automation is critically or very important to the future of IT. Yet far less than half actually use automation to any meaningful degree, with a scant 10% claiming to make extensive use of automation tools.

For example, the definition of "runbook automation" depends on whom you ask. Of those who have automated, 46% did so using homegrown software, not commercial products, casting doubt on how extensive and complete their automation capabilities really are. Yes, we're sure a few have painstakingly built sophisticated applications finely tuned for their environments. But let's face it, most are being overly generous with the term "automation software," classing homegrown scripts or tweaked open source Perl packages in the same category as enterprise IT process automation suites like those from BMC, HP, IBM Tivoli, or Microsoft Opalis. Casting further doubt on the sophistication of automation deployments, just 14% have a single infrastructure management console spanning a heterogeneous array of servers, storage systems, and network equipment.

So why aren't we getting formal here? Mostly because these suites are expensive, and they require significant care and feeding. And automation software vendors haven't convinced most potential customers that comprehensive, single-console systems provide enough value to justify the high price and lengthy deployment effort.

Once a system is in place, there's still the overhead of staff training or retooling to effectively use automation software. Many respondents don't believe their in-house skill set is up to running sophisticated automated suites.

Others blame the tools.

Research: IT Automation

Become an InformationWeek Analytics subscriber and get our full report on IT automation

This report includes action-oriented analysis and 34 charts. What you'll find:

Get This And All Our Reports

"Every new iteration of management software seems to require more extensive training than the last, and the increased complexity causes more difficult-to-diagnose problems--problems that wouldn't be there if we didn't have the management software," says one respondent. "When we have problems with management software, their support blames the changes we make, not that their software wasn't designed to anticipate that server change or application install."

Note to vendors: Where are the simpler, more easily managed products that don't take a dedicated, senior-level IT engineer to understand?

We hate to sling doom, but implementation, integration, and training problems could well worsen before they improve as we incorporate cloud services into a hybrid on-premises/online architecture.

So where's the best place to break into automation? That depends on your architecture, but let the five most-cited reasons by respondents for undertaking these projects be your guide.

Tied for first are increased productivity and improved performance. These translate to reducing the time to deploy new applications and provision servers, or increasing service availability (uptime) and reliability (minimizing unplanned downtime). Cost reductions are at No. 3, followed by improved security and meeting SLAs.

The areas that IT would most like to automate share a common theme: a high amount of manual effort. Think change and configuration management; application or system provisioning; routine maintenance, such as OS updates or patches; and identity or access management. Makes sense, yet these tasks are some of the least automated among our respondents. Atop the list of tasks most often cited as "fully automated": backups, monitoring cooling levels, and power management. When it comes to server and network provisioning or systems maintenance, fewer than 10% have gone to full autopilot. Yet that's precisely the goal of IT process orchestration software--not merely to simultaneously push out a service pack to 100 servers, but to link multiple automated tasks into an end-to-end workflow that can, for example, instantiate a new virtualized instance of a SharePoint application server at the push of a button.

One area that will give you bang for your automation buck is physical and virtual server automation. Fewer than half of our respondents automate here. But there are plentiful products, including IBM's CloudBurst, Egenera's PAN Manager, and VMware's vSphere. Better yet, these are increasingly being bundled and preintegrated with hardware management suites like those from EMC, HP, and IBM to deliver systems that can easily provision virtual servers, storage, network links, and application images.

Let Others Do The Work

Our survey also shows IT isn't taking advantage of third-party services, including outsourcers, SaaS and cloud providers, MSPs, and contract labor. When asked what percentage of their IT budgets are allocated to these providers, 36% of respondents say they spend less than 10%. And just 17% plan to increase their use of third-party services.

Lift your head up from the console and take a look at the rich ecosystem of cloud providers, managed hosting companies, and other services. These vendors are building massive facilities costing hundreds of millions of dollars, pushing the automation envelope to levels of sophistication few IT organizations even dream of.

If you can't beat them, join them. Put your entire service portfolio through some rent-vs.-own scrutiny by rating services in three areas: proprietary, regulatory, and security sensitivity; degree of complexity; and business value. Services that are common across industries or require highly specialized staff and training are top candidates for outsourcing.

Cut the waste. Look for ways to cut redundant apps. Nearly 640 business technology professionals took part in our InformationWeek Analytics Application Consolidation Survey; reducing the number of application service functions, like billing, human resources, or inventory management, delivered by pricey business software, such as CRM, ERP, or BI tools, is seen by many as a key tool for streamlining operations. In that survey, just 20% say they had not eliminated any applications over the past two years, while 70% say consolidation is as important or more important than other initiatives.

Finally, when choosing an automation suite, make sure it can handle off-site services--even if you don't employ cloud providers now, you likely will, so plan ahead.

Don't Reinvent The Wheel

Cisco Cracks The Wall

Cisco CIO Rebecca Jacoby says her team began shifting to a services orientation more than three years ago. The result:

80% of applications virtualized 57% of apps reside in automated private clouds 65% of Cisco's IT budget goes to "run the business" activities 35% of IT budget goes to "change the business" development

You can only optimize, automate, and outsource processes you fully understand, so a necessary prerequisite to bending the 80/20 ratio is applying rigor and formalism to IT governance.

The first step is recasting IT as a customer-focused business, and for most of us, that requires a change in perspective. Each IT activity must be seen as contributing to service delivery, not infrastructure operations. Now, mapping hardware and software to business services is hard work, but examining the various application, infrastructure, and support assets IT already provides, the so-called "keep the lights on" activities, itemizing their constituent dependencies and deliverables (such as applications or customer bases), and mapping these to end user services will uncover areas that can benefit from automation.

And you don't need to start with a blank sheet of paper. The IT Infrastructure Library offers a detailed, standardized service life-cycle template and a framework for service strategy, design, delivery, and operations. Alternative road maps are available from COBIT, ISO 20000, and ISO 38500. Many data center automation tools now embed ITIL processes within their workflows, meaning that adding structure and formality to service management needn't layer on bureaucratic overhead.

Bending the 80/20 barrier toward a healthy balance between service development and delivery is a way of life, not a one-time event. Such change can't be achieved with software alone --vendor claims notwithstanding. IT system management, automation, and orchestration products are valuable tools since they facilitate systematizing tasks into repeatable processes, thereby reducing errors and defects, operational costs, and execution times. But it's all predicated on defining a service catalog and understanding how process discipline can drive efficiency through automation.

Kurt Marko is an IT industry veteran.

InformationWeek: Jun. 13, 2010 Issue

Download a free PDF of InformationWeek magazine
(registration required)