IBM Storage Systems Research Envisions the Future

What is the future of storage and storage management? While all crystal balls are partly cloudy at best, one of the best bets would seem to be in IBM’s vision. When you areinvesting huge sums each year in storage research, the focus is on inventing the future rather than predicting it. The cloudiness in the crystal ball comes from a necessarydegree of uncertainty in terms of whether technical hurdles can be overcome. Whether particular technologies make the grade or not--and, if so, when--is uncertain. However, the company’s general direction seems clear, and there are many important and exciting changes ahead. Let’s take a look at some of the developments that IBM Storage Systems Research is working on to bring about the future of storage.

IBM Storage Systems Research is part of the company’s overall annual R&D investment of $6 billion. IBM research organizations work within the company itself but also participate in the scientific/technical community, with academia and government, and with clients and alliances. IBM has a long-term history of world-class research spanning many IT eras across a wide range of technical disciplines. The company has been the U.S. patent leader for 18 consecutive years.

As part of IBM Research, the challenge for Storage Systems Research is to deliver the innovation required to deal with the continuing huge explosion of data that shows no sign of slowing down while storage budgets are likely to remain relatively flat. To close the huge gap between what enterprises will demand and what they can afford, IBM Storage Systems Research is focusing its attention on a key set of initiatives: solid-state information systems, storage for clouds, archive systems, scale-out file systems, autonomic storage management and advanced storage controllers.

That is an impressive list of initiatives in itself, but we need to understand how innovation at a large IT vendor, such as IBM, differs from innovation at smaller or startup firms. The IT industry needs both, but startups typically target one product aimed at existing or emerging commercial markets with delivery slated within a few years, plus or minus. This tight focus is due to capital funding tending to be tight.

Contrast those conditions with that of a well-funded vendor like IBM. Obviously, IBM Storage Systems Research does not have unlimited funds and they have to justify its use of capital for projects. However, the company can prioritize over many projects and allocate funding as appropriate without worrying about capital starvation. (Although, at least some project managers at IBM are likely to feel that they didn’t get enough.) IBM also has the freedom to focus on long-term projects. IBM Storage Systems Research invests 40% of its budget on exploratory projects and technology not in any commercial product plan but that would meet unmet significant future customer needs as long as five to 10 years out. Obviously, startups cannot do that yet. These are the sorts of efforts that deliver significant breakthroughs that can change the face of the industry.

The other difference is in what risk means to startups in contrast to IBM. Since innovation requires coming up with something new, all innovation involves real risk.

Developing something new has the implicit assumption of the inability to achieve expectations. For startups, the risk is company survival. For IBM, on the other hand, the risk is simply that a particular investment did not pay off. But since IBM gets to employ the power of large numbers, if one project fails, that lost investment is more than recouped through those projects that succeed. And breakthroughs that push the technology envelope can also represent future product differentiation, illustrating why many larger vendors are more than willing to accept a measure of risk. (Soap box alert: That lesson should be applied to basic research on a societal basis, as well.)

One of the biggest questions about storage and one that should generate a lot of excitement is: What is the future of storage media? In other words, what technologies will eventually replace the tapes, hard drives and SSDs people really depend on today? IBM's work in this area is included in the Solid State Information Systems Initiative. This work is characterized under the label of storage-class memory (SCM), which is the company’s term for a new class of data storage and memory devices.

SCM collectively covers about 10 new technologies currently under development that will emerge in the market in a timeframe ranging from the near future (say, in two to three years) to further out (2020). SCM’s key characteristics are solid state (no moving parts, unlike disk), short access times (DRAM-like, within a factor of 10, low cost per bit (disk-like within a factor of 10), high write-endurance (which means that is it suitable for memory-like uses) and non-volatility.

There are several reasons why development of new SCM technology is mandatory if storage is to keep up with the performance and capacity requirements of future systems. The first is that the gap between the performance of disk (as measured in latency) and the rest of a large-scale, high-performance computing system is already at an ever-more-unacceptable five orders of magnitude (and recall that 105 is 100,000 times), and that gap is widening. In addition, IBM is already envisioning exascale systems that are capable of 1,018 operations per second. That requires that current obstacles in the form of energy consumption, space requirements, and the cost of the memory and storage hierarchy be overcome. The goal of IBM’s SCM technology efforts is to overcome those hurdles, and the company believes SCM has the potential to revolutionize data centers by 2020.

However, let’s not see breathless headlines that proclaim that tape (or disk or RAM or flash) is dead. All of those technologies likely have long lives ahead. Even when SCM becomes prominent on the storage stage, the older technologies will still have roles, although not the glamorous leading parts that they once took.SCM and at least several other IBM efforts deserve a separate, later discussion on their own, including these two initiatives: