Is Flash The New Disk?

Several commentators, including ESG’s Grand Exalted High Poobah Steve Duplessie, have started predicting that, in just a few years, flash will totally replace disk in primary storage, making flash the new disk, disk the new tape, tape the old tape or tape as dead as Francisco Franco. While I for one would love nothing better than to never again hear the screeching of heads across spinning media, I think these commentators, like the fashion folks that predict that heliotrope or puce is the new black, may be just a bit optimistic.

Their basic argument is that, since solid state disks (SSDs) are based on integrated circuits, their prices are subject to Moore’s law, while hard disks, being unreliable mechanical monsters, aren’t. With Moore’s law predicting that SSD density will increase 50% a year, it’s easy to extrapolate and see that, even though enterprise-grade flash SSDs cost 10 to 30 times as much as spinning disks on a dollar-per-gigabyte basis today, in a few short years flash SSDs will be down to the $100-per-terabyte level of today’s capacity-oriented drives. Folks like Steve can then assume that spinning disks as we know them will be obsolete in five to eight years, since SSDs will be down to a reasonable cost and will still be faster than even 15K RPM disks.

I have several problems with this analysis. My first problem is the all-too-common extrapolation of Moore’s Law--which says integrated circuit transistor density will double every two years--to mean that the cost per bit will in turn fall at a 50%-a-year rate. Since building fabs for ever-smaller cell geometries has been getting exponentially more expensive, chip costs don’t fall as fast as density increases.

The bigger problem is that it ignores the fact that the cost to store a terabyte of data on disk has also been falling by an average of 40% a year, a phenomenon known as Kryder’s Law. If we assume that Seagate, Western Digital and Toshiba can deliver the 20TB HAMR (Heat Assisted Magnetic Recording) or BPM (Bit Patterned Media) drives that they’re shooting for on time, the cost of commercial-grade SSDs will still cost twice as much on a per-gigabyte basis as nearline disk drives 10 years from now.

Today, people choose SSDs because they’re trying to address a real or perceived performance issue. At 20 times the cost of nearline drives, few can afford to use SSDs when only nearline performance is required. When the price differential falls to somewhere between two and four times, folks will start paying extra for new technology with no moving parts, just as people today backup to disk systems to avoid the problems they believe a less expensive tape system would present them with.

So 10 years from now, we’ll hit the magic two-to-four-times point, and the disk era should start coming to an end. We’ll have a decade of ever more sophisticated hybrid systems that use varying combinations of solid-state non volatile memory (flash, phase change or something new) and spinning nearline “trash” disks to deliver a comfortable combination of performance and capacity at a reasonable cost.

Finally, as a New Yorker and a Steelers fan, let me categorically state that black is and will always be the new black. Other colors are optional.