Cisco's Private Cloud: Pain And Profit

Cisco is not a cloud service provider, but it has an expanding suite of products under the label "Cisco Intelligent Automation for Cloud 3.1." With it, Cisco is building out a private cloud across seven major data centers in hopes that its experience will serve as a model for customers to build out their own private clouds.

Intelligent Automation for Cloud 3.1 derives both from Cisco's own data center experience and from acquisitions such as Tidal Software in 2009 with its IT automation products, and newScale in 2011 with its service catalogue, self-service portal and process orchestrator.

When it started its data center conversions in 2009, Cisco moved decisively into virtualization and found that server consolidation leads to many other changes. IT operations managers wrote scripts that automated steps in the virtual server creation process. Some integrated Cisco software with open source code. Others generated scripts for provisioning a simple database server.

"Cisco started with simple virtualization of our data center, [but as server consolidation took hold] we had a lot of tribal knowledge associated with all those different pieces," recalled Mike Myers, director of technical services for data centers and platform services (an unprepossessing title for someone who has been commissioned to convert bare metal servers and "tribal knowledge" into a company-wide second-generation private cloud).

[ For more on Cisco's cloud computing initiatives, see Chambers Reinventing Cisco, Or Recycling Tactics? ]

In February 2009, the servers in Cisco's seven major data centers were less than 5% virtualized. Cisco's IT operations will have changed that number to 95% by mid-year, and the process is still underway. When it's finished, Cisco will not only operate with heavily virtualized servers, storage and networking -- it will have also created a portal where employees automate the provisioning of their own virtual servers, set access controls and security through rules and policies, and automate many IT processes.

Myers' long name for the change is the Cisco IT Elastic Infrastructure Services. In fact, it's a private cloud, built around Cisco's own Unified Computing System servers and VMware virtualization software. Throw in a helping of open source code such as Puppet configuration management and automation packs created by Cisco for targeted environments, and you have many of the ingredients Cisco used to create CITEIS (pronounced "cities") for its employees.

In place of tribal knowledge, Cisco found it could share automated scripts across data centers to do the same things in the same way, such as provisioning a virtual server or a database server. It did not select a cloud vendor per se; it made Cisco Process Orchestrator from Tidal or the Cisco Cloud Portal with its self-service catalogue from newScale part of its own cloud suite.

That decision might seem odd since Cisco's WebEx conferencing product uses OpenStack rather than CITEIS, and Cisco even puts out an automation pack to generate an OpenStack-compatible private cloud. But Cisco joined OpenStack after its own private cloud development was well underway, and Cisco wants its suite to allow customers to create a variety of private cloud environments, not just an OpenStack-compatible one. Thus it has automation packs for its own UCS Manager for Unified Computing servers and for VMware's iSphere 5 environment.

Cisco has learned a lot, Myers said, about potential cost savings in the private cloud as well as the potential toll on IT personnel as the conversion occurs.

First, cost savings: In 2009, running a mid-sized application requiring 24 servers, 6 TBs of storage and 5 database schemas had a cost of ownership per quarter of $78,826 and would take 45 days to deploy. As the Cisco conversion progressed and by the end of 2011, the cost of ownership had dropped 45%, to $41,438 as a UCS virtualized server, network and storage environment was established. The deployment period dropped to 17 days.

A year later, total cost of ownership dropped another 24% to $31,598 as the data centers adopted workflow automation, pre-configured bundles of applications, application servers and operating systems and storage optimization. The time to deploy an app dropped to five days.

When Cisco completes the conversion by mid-year, the total cost of ownership will have dropped another 21% to $24,894 as self-service provisioning is fully implemented and other processes have been automated. The time to deploy a mid-sized app will drop to one day.

Moving from bare metal to virtualized servers provided big, easy savings. Each successive stage became more difficult, Myers acknowledged, with savings shrinking. But when it's complete, Cisco will operate 15,000 production and non-production servers and 12,000 engineering compute services servers in seven data centers as part of CITEIS.

"When we first made the self-service portal available, end users thought it was great," said Myers, adding that the addition of Oracle database schemas and middleware to the service catalogue made a huge difference in how useful the environment could be.

Some IT staffers took to the changed environment eagerly; others approached it with great wariness. The first set realized they could automate many time-consuming manual processes. The second set thought, "I've pushed this button [deploying a freshly configured application] for a long time, and now you're automating it."

"There's a very human aspect associated with a transformation like this … You had to get people into the mindset that automation didn't eliminate jobs but freed up workers to take on the task at hand. It was an education process," Myers said.

In the end, reluctant staffers were confronted with this reality, Myers said: "Your button-pushing job has been replaced. But opportunity still exists. We have the technology background and we're big enough for you to grow and be successful."

From its own private cloud deployment, Cisco is learning what its customers will need as they strive to achieve self-service operations and 95% virtualization. The process is not easy, but the total cost of ownership figures, if borne out by others, indicate it's worth the effort.

Cloud Connect returns to Silicon Valley, April 2-5, 2013, for four days of lectures, panels, tutorials and roundtable discussions on a comprehensive selection of cloud topics taught by leading industry experts. Use Priority Code DIWEEK2 by Jan 26 to save up to $500 with Early Bird Savings. Join us in Silicon Valley to see new products, keep up-to-date on industry trends and create and strengthen professional relationships. Register for Cloud Connect now.