2012 State Of Storage: Year Of SSDs

The growth rate of enterprise data is finally stabilizing, according to our 2012 InformationWeek State of Storage Survey. Finally, piling on disk capacity, upgrading storage area networks, and adding data-reduction technologies aren't atop most respondents' to-do lists. That doesn't mean storage administrators are kicking back, though. Our respondents still worry about having the resources to support mission-critical applications and are more concerned than ever about keeping data safe and sound.

Storage pros are responding to these challenges with creative strategies, such as combining solid-state storage for high throughput needs; cloud services for backup, archiving, and disaster recovery; scale-out storage architectures for cost-effective, highly resilient capacity growth; and data encryption to protect information both on premises and off. The mainstreaming of 10 Gigabit Ethernet and data and storage network consolidation are adding their own twists.

On the business side, we're seeing signs of an improving economy, and, for IT, nowhere are these positive signals stronger than in the storage market. Despite some natural disasters that battered the component supply chain, 2011 was a year of robust growth in the storage business. IDC's most recent Worldwide Quarterly Disk Storage Systems Tracker found that, in the third quarter of 2011, storage vendors posted year-over-year revenue gains of 10.8%, to just under $5.8 billion, while capacity of total shipped systems rose 30.7% year over year. Two of the largest pure-play storage vendors, EMC and NetApp, saw 15% to 20% revenue growth last year. On Wall Street, shares of drive manufacturer Seagate jumped more than 40%.

In the enterprise, the 313 business technology professionals we surveyed, all of whose jobs involve managing, operating, or buying storage systems and services, suggest that we're entering a period of steady but manageably linear storage growth, with most reporting rates of less than 24%. Technology is helping, too. Commodity SATA drives now top out at 3 TB, so adding capacity is less of a challenge.

In fact, we're entering an era in which performance--specifically, I/O throughput--is more important than capacity in a storage infrastructure strategy. Notwithstanding price spikes resulting from temporary supply disruptions, such as Thailand's floods, hard disk capacity has gotten so cheap that the biggest return on your storage investment comes by boosting application performance. In storage, this means increasing I/O throughput and decreasing latency.

A major driver of this change in emphasis is the falling price and increasing capacity of solid-state disks, but it's by no means the only one. The pervasive use of virtualized servers improves resource utilization, but at a performance cost, since many applications share a single system's network and storage interfaces. For IT, that means you need not be running a high-end transaction-processing application to tax the throughput of mechanical drives; 10 or 20 SharePoint servers could expose bottlenecks.

Just replacing a mechanical disk with an SSD while sticking with the same controller-based storage array design is far from an optimal solution. Jonathan Goldick, CTO of Violin Memory Systems, says the problem with hybrid setups--think trays of SSDs commingled with HDDs--is that the controller architecture is designed for spinning disks. "Ten SSDs will saturate any controller," Goldick says. That reality is spurring interest in specialty, purpose-built solid-state storage systems.

Research: State of Storage 2012

Our full 2012 State of Storage report is free with registration.

This report includes 44 pages of action-oriented analysis, packed with 37 charts. What you'll find:

Get This And All Our Reports

Consolidation Continues

Direct-attached storage remains the most broadly deployed means of connectivity. Fully 92% of respondents still have some DAS systems hanging around, and 23% use DAS for half or more of their storage needs, virtually unchanged from last year. Fibre Channel remains the most pervasive connection technology, however, with 38% of respondents putting more than half of their storage capacity on FC, up two points from 2011.

Pundit predictions of the death of Fibre Channel over Ethernet are much exaggerated: We found 38% of survey respondents have deployed this convergence technology, up 10 points in the past year, with an intrepid 8% using FCoE for half or more of their storage needs.

Despite the inevitability of Ethernet becoming dominant in storage networks, our InformationWeek Data Center Convergence Survey found that a mere 8% of FCoE users have fully eliminated native Fibre Channel, although an additional 63% are planning to. Only 29% expect to use native FC in the long term. Despite what boosters say, Fibre Channel's prognosis is grim. However, since the turnover in storage networking equipment is relatively slow, we believe it'll be a factor for at least five years, at which point we'll have 40-Gbps or 100-Gbps Ethernet to the edge.

DAS is still common, particularly among smaller enterprises, because of inertia, but its days are also numbered. Most respondents to our storage survey have either already consolidated to centrally managed storage systems or expect to do so--although for many, their plans are still vague. We expect the share of DAS to drop as distributed, cloud-like file systems and scale-out architectures catch on. That's a vital concept: There have always been two design approaches for adding resources or capacity to an IT system: scaling horizontally, by adding nodes, or scaling vertically, by augmenting internal resources. In the world of storage architectures, these are known as scale out vs. scale up. Scale-up systems are still the most common in big companies, and their use has grown apace with the rise of server virtualization and the attendant consolidation of compute and storage workloads.

Scale-out storage systems work differently. Instead of adding more disks and controllers to a single array, IT can increase capacity simply by placing more nodes in a networked cluster. Scale-out appliances, some with embedded hypervisors, represent a threat to big, centralized, high-profit-margin arrays from vendors like EMC, Hewlett-Packard, IBM, and NetApp. And central management software and features like storage virtualization and automated load balancing make an assortment of disks and solid-state storage look like a single resource pool.

That's a good thing, too, since the last thing IT needs is more management complexity, no matter where it resides. One-quarter of our respondents cite insufficient tools for storage management as a top concern, which is understandable since only 31% have dedicated storage teams. But a lack of management is just one factor contributing to data protection concerns.

Reliability And Security

When it comes to the storage technologies most used and valued by our respondents, reliability and security trump bleeding-edge features. Not that IT's standing still--we asked which new storage technologies respondents have deployed and saw a general increase across the board.

However, a couple of spikes in the past year indicate that data availability is more important than ever. For example, 67% of respondents use data replication, up 10 points in a year. Other data-preservation technologies, like disk-to-disk backup and snapshots, also rank near the top of our list of 15 storage technologies currently deployed. Another noteworthy jump occurred in the share of those using data encryption, now 55%, up eight points in the past year. Similarly, when we asked about use of backup tape encryption (yes, 43% of respondents still widely use tape), the numbers were up again this year. Fifty percent encrypt some or all of their tapes, while the portion of those with no plans to encrypt dropped nine points.

These results aren't flukes, either. When we asked respondents to rate the most important factors when evaluating new purchases, once again, data replication and encryption came out on top, both cited by more respondents than a year ago, confirming that data protection is top of mind.

As for deduplication and data compression, while they're used in production by 45% and 52% of respondents, respectively, fewer than one-third consider them important evaluation criteria, down from our last survey. Our take is that IT considers dedupe and compression relatively mature and undifferentiated technologies that are--or should be--standard features in any enterprise-class storage system. Vendors expecting them to tip the competitive balance will likely be disappointed.

So what will be a selling point?

This year's survey shows that the age of solid state is upon us: 20% to 23% of respondents, depending on system type, already use SSDs. Although they aren't yet considered a key product evaluation criterion, this will surely change as prices drop and performance improvements stack up, and as products optimized for solid-state memory gain visibility. Once storage system capacity, scalability, and reliability become baseline expectations, throughput and latency will inevitably become differentiators.

Big Data, Big Problems?

Call us pessimists, but while storage capacity growth rates are down from last year, the sources of expansion are changing--and in a way that could be dangerous if we're not careful.

In aggregate, structured and unstructured data are now neck-and-neck as the leading growth sources in Tier 1. At Tiers 2 and 3, however, it's no contest: Unstructured data is the culprit, voraciously consuming every terabyte we throw at it. Fully 77% of our respondents list some form of unstructured data as the leading source of Tier 2 storage growth; that's up from 70% last year. For Tier 3, the share is an equally commanding 71%.

The rapid growth of unstructured data in Tier 3 is also apparent when looking at retention rates. The top four data types retained for up to five years are unstructured. We did see one small but encouraging sign of growing discipline in storage management--specifically, a drop in the percentage of respondents with no retention policies at all. This indicates that formalized information management, once confined to databases, has spread to unstructured content in the majority of companies--and just in time, too.

The fact that we're keeping piles of unstructured data for years and years brings up a question: Where do we put it? For some, the cloud is the answer. The share of respondents saying they have absolutely no plans to use cloud storage dropped eight points this year, while the number using it in some capacity, whether for email, archiving, or backup and recovery, increased by eight points. One respondent sums up the main driver nicely: "Because of the cost of maintaining and managing storage, I am looking more to the cloud for nonessential data storage and backup."

Still, that protective instinct is at play. Roadblocks to greater cloud adoption remain largely unchanged from last year: security, reliability, and performance. We'll be watching to see how cloud vendors go about assuring IT that they can protect our data. Meanwhile, they're clearly getting their cost-effectiveness message across: The share of respondents labeling cost a major concern dropped nine points. Since we didn't see any big cloud storage price wars in the past year, this could reflect IT's growing understanding of the model's benefits.

5 Smart Steps To Take

2012 is shaping up as a good year to address storage problems that have been allowed to languish, since the portion of our respondents citing budget constraints as a top concern dropped six points this year.

As you refine your storage plans for 2012, here are five areas to address:

>> Develop a solid-state strategy: Merely replacing high-end HDDs with SDDs leaves potential performance gains on the table. That's because conventional storage arrays are designed to work around the inherent deficiencies of a mechanical system. So think beyond solid-state disks to solid-state systems. Look for I/O-bound applications that can benefit from orders-of-magnitude improvements in throughput and latency, and where there's more bang for the buck in adding speed vs. adding capacity.

>> Consolidate intelligently: Most IT shops are still sitting on a large base of direct-attached storage, but the widespread adoption of server virtualization has many planning consolidation projects. Clearly, the new generation of scale-out storage hardware couldn't have come at a more opportune time. Finally, IT doesn't need to blow big bucks on a huge FC storage frame to consolidate effectively. Think about whether some applications that are already virtualized, particularly those with large and rapidly growing capacity requirements, might work better on an Ethernet-based scale-out system.

>> Continue data/storage network consolidation: With the price of 10-Gbps Ethernet dropping below $200 per port and integrated 10-Gbps interfaces becoming the norm on new server and storage hardware, it's time to get serious about making Ethernet your storage backbone.

>> Get serious about encryption: Data security remains most cloud users' biggest concern. But data loss isn't an issue only when using outside services; it's now the leading storage concern in general, and encryption has become one of the top technologies buyers look at when evaluating new ways to protect information. But it's not enough to just get some self-encrypting drives and hope for the best--you still need to worry about key management and protecting data in flight to and stored in the cloud. So get a holistic encryption plan in place.

>> Develop a cloud storage strategy: Those not using cloud storage and with no plans to do so should be freeing up funds for these projects. Cloud storage is mature, reliable, and particularly well suited for data backup, archival, and disaster recovery scenarios. Security doesn't need to be a deal breaker. Do an analysis of total cost of ownership going out five years and use providers selectively.

Continue to the sidebar:
Solid State 101: Where To Deploy SSDs Now

InformationWeek: Feb 27, 2012 Issue

Download a free PDF of InformationWeek magazine
(registration required)