The Case Against Orchestration

WHITHER ORCHESTRATION?

Automation and Orchestration
Vendors are focusing on orchestration and automation of their infrastructure to extract pounds instead of pennies from their customers.

Implementing Automation
If you decide that’s it time to try automation, let's consider what options, tips and plans you could make for a successful deployment.

In Automation and Orchestration, I outlined the case for Orchestration. Now I want to answer "Why now? What's different?" After all, management platforms have failed to deliver automation and orchestration for the last 20 years at least. Why would today's software be any more likely to actually work? Can we try for operational nirvana once again?

It's true that the operational management road is littered with failed attempts. HP OpenView, BMC Patrol, Microsoft SCVMM--all have been over-complicated, consultant-rich and expensive failures for all but the rich few. Dozens of other failed companies have had varying levels of success at attempting to automate a single process, much less orchestrating them into a single whole.

I believe that the failure factors can be reduced to a basic set of points: technical limitations , people problems and management will.

Technical Limitations
In a competitive market, vendors have always attempted to use their existing customers as cows to be milked of cash. Once you purchased a vendor's products, then the vendor aimed to lock the customer into its own ecosystem. As an example, Cisco makes its own network management platforms that are universally reviled by customers for poor functionality, slow performance and lack of features. While claiming that CiscoWorks had better access to developers and would be first to market with feature support and the best integration, it never really delivered on the promise.

The other side of this picture is that supporting third-party management platforms for your products is difficult. Consider Microsoft's SMS/SCVMM management tool set, which promised support for server hardware for automatic detection of drivers and patches, yet remained years behind the release cycles. In the end, SCVMM is used only to manage Microsoft products and still lags months behind the release cycle of any new products. Another example, Cisco CS-MARS, supported third-party firewalls on acquisition but ditched support later because vendors couldn't, or wouldn't, co-operate and integrate their security products.

A lot of equipment doesn't have the resources to run extra software. Many vendors struggle to have stable products at all, and the very idea of adding more functions that would increase costs (and reduce profits) and might introduce more bugs is not good business. Therefore, vendors focus on getting their core functions working and nothing else.

Finally, the other limit is the constant development of new technologies. As new operating systems, switching silicon and storage functions arrive, it means that management tools are quickly obsolete or holding back adoption.

People Problems
Although orchestration is a technology challenge, ultimately the people involved affect the decision making process. The lack of multiple disciplined people means that resources to build and maintain the orchestration are very limited in supply For example, storage networking was so complex that only a focused resource could comprehend it. Moreover, the people with the vision to lead such deployments are even harder to find. That said, the use of hypervisor virtualization has led to convergence among the network, server, storage and OS teams that provides some hope for the future.

And, finally, job security. Some people will feel threatened rather than embrace the challenge to learn new skills. Their role will move from routine tasks to routine analysis and strategy review. This can be hard to perceive.

Management Will
The practical reality of modern IT is that the demand for uptime means that deep change to an existing infrastructure is impossible to achieve. The process of ongoing incremental change (as espoused by ITIL and PRINCE2) means there is no opportunity to orchestrate the entire data center process and take deep risks with live services. Even the opportunity to save large amounts of money and time is never going to be enough to satisfy the ITIL change prevention processes on systemic risk.

And while it's tempting to consider implementing automation for smaller areas, in practice, orchestration is an all-or-nothing business. That's why cloud computing is likely to be popular: Someone has already implemented the orchestration, and you can just "move in" with your software. This is perceived as incremental risk, not as systemic risk.In the last two years, we have seen VMware win a lot of market share by delivering limited orchestration capabilities. The key success factor is that VMware has abstracted the operating system from the underlying hardware, such as CPU, RAM, storage array and networking. The ability to move Windows and Linux servers from servers to server is hugely cost-effective. It seems that these successes have driven a sea change in vendor attitude to "allowing" orchestration unless they are frozen out of a collaborative market.

This business drive, coupled with the success of "Web 2.0" platforms, means that most vendors now developing, publishing and supporting APIs that allow external configuration. For example, Cisco Nexus switches expose a complete XML API using upcoming industry-standard NETCONF/YANG; storage arrays (including EMC) have proprietary and open (VAAI and VASA) standards; Mware has several APIs for Java, C# and Perl, as well as Web APIs.

In addition, the rise of decent programming skills and tools is at the point where useful software is now much cheaper to develop and deliver.

The Case Against Any of This
Let's summarize arguments against orchestration (but you recognize more than a few traditional negative change problems, too).

And the list goes on. The reality is that orchestration is a big thing, requiring big changes in organizations, changes in skills and team disciplines. And it's risky. Mucking around with live services on the crucifix of change control and unrealistic SLAs make sweeping changes difficult or impossible. Knowing what you are up against, is it worth it? More importantly, what can we do to raise the chances of success?