IT Pros Mistrust Cloud Performance Claims

A survey of companies that use the cloud to deliver business applications shows that nearly three out of four of them think their cloud providers are hiding performance problems from them.

The survey of 704 senior IT pros commissioned by Compuware, a provider of application performance management technology, shows that 73% of respondents believe that companies hide performance shortfalls with application delivery. Seventy-nine percent believe that cloud provider service level agreements (SLAs) are poorly written and don’t adequately explain what assurances providers actually make. Survey respondents view SLAs as too simplistic.

"Companies fear that infrastructure and platform-level problems are being hidden behind hollow guarantees, with little incentive for cloud providers to give any greater levels of transparency," a whitepaper based on the global survey states. "Vanity SLAs provide a false sense of security and do not capture the true customer and end user requirements that properly mitigate business risks with cloud infrastructure."

Simplistic SLAs, the report indicates, are those that emphasize "availability" of the cloud application without accounting for many variables in application delivery.

For instance, a company may be assured by its cloud provider that its website will be available year-round, but performance may suffer if there’s a rush of business for an online retailer around the holidays, at a floral site prior to Mother’s Day or on a health insurance site during open enrollment for coverage.

A site could easily find itself in a position where it has been available all year long but suddenly fails when it really counts, "yet the cloud provider has still met the minimum availability SLA," the report states.

The essence of the problem in such instances is the difference between "availability" and "performance," said Thomas Mendel, an analyst with Research in Action, which conducted the survey for Compuware.

"Most cloud providers will say they have 99.999 or whatever percent availability. That’s not the problem," Mendel told Network Computing. "Many of them will not give you a performance guarantee because there are very few companies who can actually guarantee performance over the long run."

Performance shortfalls could result in customers getting on the site, but their transaction not being completed, meaning Mom doesn’t get her flowers and the customer doesn’t get signed up for the insurance he or she seeks.

The whitepaper advises companies to insist on flexible SLAs that are based on their needs instead of the cloud providers’ reporting requirements. Additional metrics that should be disclosed include end-user response time, the error rate on transactions, and capacity planning.

In addition, Compuware’s survey reveals that cloud customers believe that platform-as-a-service (PaaS) and infrastructure-as-a-service (IaaS) providers are not always forthcoming about whether service problems are related to failures in the provider’s own infrastructure. The report advises companies to regard cloud providers as an extension of their own IT departments and hold them accountable for problems that impact the company’s bottom line.

However, that can be problematic, too, Mendel said, because while a customer can see into its own data center, it has little idea what’s going on in the cloud.

"With this end-to-end SLA, it is very difficult to figure out if something goes wrong, who’s to blame? And there’s no one who offers a good solution to bridge that gap," he says.

The survey showed that 62% of respondents said troubleshooting application performance is difficult because they lack the necessary visibility into the cloud environment. Compuware notes that some cloud providers are starting to offer premium services to improve that visibility, such as guaranteed IOPS, but warns that those kinds of options are just point solutions and may not provide the complete visibility customers really need.

The survey also highlighted another problem: Performance issues traced to adjacent tenants in the cloud environment, which the report describes as "noisy neighbors" that may be monopolizing resources in the cloud. Sixty percent of cloud customers surveyed said they worry about that.

There is hope for improvement. As cloud deployments get more complicated and customers demand more solid SLAs, the cloud market will be increasingly dominated by the larger cloud providers with the capabilities to deliver the necessary level of availability and performance, Mendel said.