Special Issue--IT Automation: IT Process Automation

The Jetsons provided a utopian image of future automation--George's workday encompassed pressing a single button, then relaxing in his comfy chair. Although automation has made strides over the years, we're far from one-button technology when it comes to automating enterprise IT processes. Most process-automation tools have required massive development efforts and custom-built software adapters, but the vendors whose three products we tested--Opalis, Opsware and RealOps--are ushering in new tools that offer prebuilt automation and can be rapidly deployed. In our last special report on this topic (see "Autonomic Computing: Vision vs. Reality"), we said dynamic functions, such as configuring services, continual optimization, self-monitoring and interaction with the rest of the environment, were critical for transformational autonomic computing. This is the ideal vision, and though many vendors are working to meet those expectations by offering prebuilt workflow and integration, none of their products can dynamically adapt as the environment changes.

However, the products we tested--Opalis Integration Server, Opsware Process Automation System and RealOps AMP--are valuable for consolidating process workflow, as well as integrating and automating routine tasks. All contain prebuilt automation for repetitive tasks and don't require much effort to build processes using those automations. Some customization capabilities are complex, but the prices are reasonable considering the significant value these tools provide.The products we examined range from $100,000 to $500,000. Sure, the price is significant, but how much is your organization spending on complex processes, such as provisioning, order entry, service activation, new user activation and deletion? These are probably costing you much more every year.

Fitting In

Because many IT software vendors still rely on proprietary APIs for tight integration among their products, there are problems with getting those products to work with IT automation systems. Optimization is heavily dependent on the user's construction of policies and workflow, as the products have little ability beyond error checking to validate processes. In those terms, these products are like a toolkit that lets IT operations customize how they want automation to work in their environments and come with a boatload of prebuilt automations to assist setup. With some assistance from the vendors, customers can implement new policies and rules on how to communicate with their environments.

Considering the need for smooth integration, our first concern was the breadth of the integration support for existing network-management applications and the existence of add-on modules for COTS (commercial, off-the-shelf) software. All three products we tested offer a laundry list of application support. To integrate with our test environment, Opalis Integration Server requires the Hewlett-Packard OpenView Interconnect (OVI) module, while Opsware Process Automation System communicates with OpenView Operations (OVO) through the API. Opalis works directly with Remedy ARS, however, while Opsware requires the Remedy midtier server. From an adapter perspective, RealOps and Opsware focus on fault, configuration, performance and service-desk apps. Opalis adds a breadth of support for Microsoft applications and other application and job-scheduling integration.Because many organizations need some help defining processes, the vendors supply process-automation tasks that can be used as a starting point. Processes always change over time, so you must be able to modify the workflow. To answer this concern, RealOps' AutoPilot, a workflow packet, focuses on network automation, while Opalis' Policies component and Opsware's Accelerator Packs concentrate on server and network automation. These functions automate many of the routine tasks that operations may perform around IT systems. All have easy-to-navigate click-and-drag approaches for creating workflows.

In addition, these products must communicate with their environment--including IT managers. Since few organizations want to trust the systems implicitly out of the gate, the IT automation software should report on the status of the processes that are automated, as well as on any exceptions or problems encountered along the way.

Opsware is the only vendor offering a visually guided interface through its native dashboard--a critical feature for tasks that still require manual intervention. Opsware provides a management dashboard and audit trail of automated resolution steps for complete ITIL (IT Infrastructure Library) incident- and problem-management visibility. In addition to getting out-of-box reports like MTTR trending and cumulative ROI, users can set up custom reports. RealOps automates higher volume processes where step processing isn't practical and can report results using standard XML. Opalis has a Policy log that lets users view both real-time and historical information about the processes that have been executed and any problems that were encountered.

All of these bells and whistles come with a price, so does the benefit of automating the processes outweigh the time and complexity of installing, configuring and maintaining the software? Large enterprises with complex processes will gain the most value from these applications, given the scale of potential savings. However, one of the primary objectives of IT process automation is to reduce the complexity and cost of IT systems and reduce the need for human administration. All the products require time and skill to set up--this is best left to the vendors or qualified systems engineers.

For larger environments, we estimate several weeks to deploy and integrate the applications. The time to create the processes will depend on the number and complexity of processes to integrate. With all three products, organizations still must worry about version numbers and how application upgrades will affect the IT automation software. Once the systems are deployed, creating processes and workflows are straightforward.In critical enterprise environments, you must ensure that the systems automating them are highly available, redundant and support a distributed architecture. Organizations should expect vendors to deliver in this area. RealOps uses a peer-to-peer grid system that maximizes its benefit in a distributed environment. AMP components can assume all the capabilities of the process workflow, integration or reporting aspects if one of the nodes in the AMP grid fails. This is extremely important in highly available environments. Opsware uses a cluster environment for high availability. Larger distributed environments should test the performance of the products for high volume processing, as a peer-to-peer architecture such as RealOps may perform better than a publish-subscribe architecture such as Opalis.

Architecture

On the surface, the products we tested have similar architectures (see diagram, page 73). They offer software adapter modules that connect to your applications. Typically, these software components are installed on the application to which you are connecting.

The adapters then connect to a hub, similar to a middleware bus, where information can be exchanged between the applications. This hub also acts as a distribution point for process and workflow logic, which resides on top of the hub, where processes and automation can be applied to the information exchanged.

From a design perspective, the products have a visual designer component that lets IT construct workflow and process design. Here, processes can be designed, tested and configured to run based on conditions or time. The products also offer reporting tools to provide a window into the environment and status on executed processes.Cautious Optimism

Although the full promise of autonomic computing is still on the horizon, these vendors are creating platforms that IT can integrate with external surroundings, based on open standards. All three products we tested fit very well into change-management, CMDB and virtualization projects. Each is also very compelling in terms of letting IT staff concentrate on complex problems and say goodbye to routine tasks. The more processes and standards you have in place, the greater impact these tools will have on your organization.

All three products provide attractive capabilities for automating routine and critical functions performed in most IT environments. And all are much less complicated than most EAI and middleware tools.

If you're considering these applications, select one process or process area to automate to test the tool's value. Before selecting a product, use our ROI worksheet to determine how much time you spend on the manual process and what the impact would be if it were automated. We may not experience the one-button workday of George Jetson, but these products could make your work quite a bit easier. n

ArchitectureOn the surface, the products we tested have similar architectures (see diagram, page 73). They offer software adapter modules that connect to your applications. Typically, these software components are installed on the application to which you are connecting.

The adapters then connect to a hub, similar to a middleware bus, where information can be exchanged between the applications. This hub also acts as a distribution point for process and workflow logic, which resides on top of the hub, where processes and automation can be applied to the information exchanged.

From a design perspective, the products have a visual designer component that lets IT construct workflow and process design. Here, processes can be designed, tested and configured to run based on conditions or time. The products also offer reporting tools to provide a window into the environment and status on executed processes.

Cautious Optimism

Although the full promise of autonomic computing is still on the horizon, these vendors are creating platforms that IT can integrate with external surroundings, based on open standards. All three products we tested fit very well into change-management, CMDB and virtualization projects. Each is also very compelling in terms of letting IT staff concentrate on complex problems and say goodbye to routine tasks. The more processes and standards you have in place, the greater impact these tools will have on your organization.All three products provide attractive capabilities for automating routine and critical functions performed in most IT environments. And all are much less complicated than most EAI and middleware tools.

If you're considering these applications, select one process or process area to automate to test the tool's value. Before selecting a product, use our ROI worksheet to determine how much time you spend on the manual process and what the impact would be if it were automated. We may not experience the one-button workday of George Jetson, but these products could make your work quite a bit easier. n

When Process Automation Makes Sense

Automation can have a transformational impact on an organization, reducing costs and improving efficiency, but is it right for you? Generally, if you aren't using a standards-based process methodology, you aren't ready for these applications. However, they might make sense if you're ready to take the next step in IT management:

Review Scenario: IT Process Automation Tools

We wanted to test products that are pure-play autonomic computing, process automation, or run-book automation software. All existing connectors to the applications tested had to be developed as did prebuilt processes. Filling the bill were Opalis Integration Server, Opsware Process Automation System and RealOps AMP. Optinuity and Network Automation did not respond to our invitation to be included in this review.

PARTICIPATING VENDORSOpalis, Opsware, RealOps

TESTING SCENARIO

We created a BMC Remedy Service Desk incident based on the receipt of an event from HP OpenView that should be prioritized based on device service levels queried from a Microsoft SQL database. The products had to scan the host system for installed products and compare the results to an Excel spreadsheet of "approved" application processes, then send an e-mail alerting the operations staff of a problem if a discrepancy existed. This process involved commercial off-the-shelf products and data sources typical to most environments. • Usability: Rated in terms of how easy it is to create and modify specific workflows based on organizational requirements and to implement automated policies based on defined processes.

RESULTS

Opsware Process Automation System stands out as a leader with a good balance of features, cost and scalability, while Opalis Integration Server was the easiest to use of the products tested. RealOps AMP offers high reliability with its steel-clad architecture. All three are suitable for large organizations; because of its ease of use, Opalis as also well suited for midsize companies.Review: IT Process Automation Tools

Once the IT process-automation products are installed, most users will be ready to design processes after just a short training class or scouring the manuals. All three products we tested--Opalis Integration Server, Opsware Process Automation System and RealOps AMP--can easily automate processes that involve manual steps through a simple visual interface where no programming language is required.

Although RealOps' Auto-Pilot workflow module is great for network tasks, Opsware Process Automation System and Opalis Integration Server are much stronger in the application-automation arena. Both have many more prebuilt server- and application-automation tasks. Opsware's server-automation packs are especially useful in large, distributed-application environments focused on application-incident detection, tracking and resolution. Opalis' history with job scheduling and dozens of templates also help jump-start application processes.Opsware has a big advantage with its visually guided process engine. Users can fully automate a process or allow operations transparency into each step. Through the dashboard, users can move to the next stage of the process with the click of a button. Because these tools can transform the way processes are managed, it's important that users feel comfortable with automation, especially in early stages.

Opsware's ability to produce a set of extremely polished documents that described all our processes was impressive. This documentation helps in regulatory reporting, such as that needed for Sarbanes-Oxley and ISO 20K, and in eliminating the need for costly process-documentation exercises that produce rarely used guides. Although the products from RealOps and Opalis also consolidate and store processes in a central location, exporting the output from them is awkward.

Opalis has an adapter for HP OpenView OVO, but we needed assistance from the vendor to be able to listen for events, as we were just able to insert events. We worked through this issue during our testing fairly quickly. If you're just running OpenView Network Node Manager, you'll also need HP OVO (OpenView Operations) and HP OVI (OpenView Interconnect) for the integration pack to work. Integrating with Remedy ARS (Action Request System), however, was a snap as it used an API. Opsware's setup requires the installation of Remedy's midtier--it communicates using Web services, not the API. This could be an advantage if the APIs were to change substantially with new versions of products.

RealOps defines its adapters as "base," meaning they communicate using protocols, or "product," meaning they communicate with APIs. RealOps' setup communicates with Remedy using the API and listens for OpenView events using the OVO API and using its SQL adapter to connect to other data sources.

As far as process development, both Opsware and Opalis use product-specific templates and workflows. RealOps uses the BPEL model for process flow and has its templates organized around functions. A service-desk-process workflow, for example, lets the process designer focus on the functions and not the specific product. The translation to Remedy, or any other service-desk tool, is done at the adapter layer. For organizations that want to share workflows or have multiple service-desk implementations, this will help them create standard processes across groups. Also, when users change tools they don't have to change the process flow in RealOps--they just need to change the adapter. In Opalis' and Opsware's setups, you must change the processes and the adapters.As these products become more common, RealOps' use of BPEL-based modeling standards will help limit the complexity that could develop with an application of this type. With only 10 activities, such as call adapter, script and pause, the interface is powerful yet not overly complex. Both Opalis and Opsware offer more options for users, but in larger environments, this can become unruly as the systems are used by different individuals using different standards. RealOps is better for tightly defined processes that can fit within the BPEL model.

The Best Fit

Opsware Process Automation System stands out as a leader with a good balance of features and scalability. Since Opsware also has configuration-management tools, there are some compelling synergies with configuration management across the network, application and storage infrastructure. With some of the complementary Opsware products, such as SAS and NAS, Opsware Process Automation System can extend those capabilities into broad configuration-management capabilities. Opsware's documentation ability and visually guided processes also are key differentiators for environments that require these features and provide full transparency to IT operations.

RealOps shines for large, high-volume environments that encompass large distributed infrastructures, such as telecommunication service providers and large financial service providers. The standards-based approach to process modeling will scale well in environments that distribute the management of the processes and help maintain control of the tools, and they offer steel-clad architecture. Although building some of the initial workflow may take just a bit longer in RealOps and XML knowledge is required, the power and flexibility of the architecture and its ability to execute hundreds of thousands of processes without a performance hit is a big plus.

Opalis's product is the easiest to use. And Integration Server is the best fit for organizations that have application-centric process automation needs. Its focus on prebuilt application templates for Microsoft applications are added value for many organizations.Opsware Process Automation System

Although Opsware Process Automation System offers the most robust set of features, large distributed organizations should first test the scalability of their architectures. For extremely high-volume transactions and native high availability, Opsware relies on clustering. This might work well for many enterprises--but organizations that need a robust, high-availability architecture and do not need extra bells and whistles should consider RealOps instead.

Opsware has a nice balance of Accelerator Packs and Integration Packs. Accelerator Packs are collections of preconfigured flows and operations for common IT infrastructure. Integration Packs are collections of preconfigured operations for interacting with systems-management tools, such as those for provisioning, monitoring and ticketing. Opsware has hundreds of preconfigured flows and operations that address common problems on infrastructures that use Cisco routers, Linux, Unix, Windows and VMWare. Opsware provides a good balance of integration support and prebuilt workflow for most enterprises. Its prebuilt application- and network-specific workflow can be a big help to organizations getting started in process automation.

Some of the product's other key features help it to stand out. One unique feature we really liked is the ability to assign a dollar amount to individual workflow-process components. This is useful in an environment where IT charges for IT services.

Opsware Process Automation System also is the only product that lets workflows run in one of two modes--automated or guided. Automated workflows take place without staff intervention; guided ones use a wizard-based interface to step operators through diagnostic and remediation procedures.Process documentation is implemented very well in Opsware Process Automation System. Workflows are autodocumented and autoaudited. Once workflows are authored, the OpsForce component automatically generates documentation for use in audit-compliance checks. For each workflow run, OpsForce reports on when it took place, what steps were taken, who ran the workflow and what the workflow result was.

Finally, Opsware's business-intelligence dashboard is the leader of the pack. Incident, problem and change-management visibility are included out-of-box. Useful reports for MTTR trending, cumulative ROI, most frequent resolution flows and incident alerting are all there.

Opalis Integration Server

Organizations looking for ease of use should check out Opalis Integration Server. Much of the application's complexity is hidden from users, making process setup and automation a breeze. Overall, Opalis' product was the easiest to use of the three applications we tested. Creating workflows is a snap and Opalis can tell which fields in Remedy are available and required for data entry. An Opalis user with limited knowledge of the underlying applications could easily create a powerful workflow using the interface.

Opalis sent us a virtual machine with Integration Server installed and preconfigured adapters to HP OpenView and Remedy. We immediately ran into some setup glitches, however, as Opalis also needs OVI to communicate with OpenView. Once we installed OVI, we discovered that Opalis can insert events into OpenView, but we needed help from the vendor to detect events created within OpenView. This forced us to insert an event into Opalis manually to launch the automation. Of the products we tested, Opalis was also best dealing with application integration. Its product supports a number of Microsoft business applications, including SMS, Virtual Server, Active Directory and Operations Manager, as well as VMWare VirtualCenter and the usual suspects of IT operations-management tools and service desks.

Opalis lets you perform file-management and text-editing activities on Unix machines through its Integration Pack for Unix. It also provides integration over SSH, SNMP and other open cross-platform protocols, such as SOAP/HTTP Web services. Also, the product's integration packs support both Unix and Windows versions of systems-management products, such as HP OpenView Operations. However, it cannot communicate directly to HP's OpenView Network Node Manager.

Opalis lets you build manual checkpoints into processes, but the method is extremely awkward compared to that employed by Opsware Process Automation System. Getting information about automated and manual policies is challenging for administrators.

For high-availability environments, however, Opalis is not a great fit. Its publish-subscribe model in the message bus and architecture are not the best for environments that require near 100 percent uptime. Opalis offers peer-to-peer communication between the runtime Action Server components for high-availability and failover and supports database high-availability for many distributed environments, but still falls short.

RealOps AMPThe real power of AMP is in its architecture. The product's peer-to-peer grid setup lets each physical node of the system act as an independent entity, allowing maximum availability and performance. The adapters are controlled using the Configuration Development Peer. The CDP is responsible for configuring the adapter-grid relationship and how processing and failover are handled.

All design work from the studio is exported to the grid, where detailed workflow versioning can be retained. Organizations that are looking for full traceability in workflow changes and control will find this feature invaluable.

Although RealOps touts its abstract workflow for application-process design, the workflow still is dependent on the actual applications. You could build an abstraction layer to remove the application specifics from the workflow, but this capability is present in the products from Opalis and Opsware and isn't worth the added complexity.

In addition to the grid-architecture strength, RealOps gives you access to just about everything. Designers can get under the covers and manipulate functions like application communications, extending the product to support in-house applications, and configure a number of architectural options. This is a big plus for organizations with applications that are not supported through commercial, off-the-shelf adapters.

However, this flexibility has a downside. RealOps AMP was the most complex product we tested. This isn't surprising given the additional capabilities with the architecture, but may be a problem for many midsize IT organizations.Finally, AMP's lack of a dashboard is a big detractor for RealOps. Integrated reporting and metrics are critical for these applications and Opsware Process Automation System specifically has a big advantage here. AMP can export data over XML to many reporting applications, but a native dashboard is critical.

Michael Biddick is an NWC contributing editor and executive vice president of solutions for Windward Consulting Group, a firm that helps organizations improve IT operational efficiency.