When it comes to TCO, solid-state drives are a better deal than HDDs.
With all the hype and counter-hype on the issue of solid-state drives versus hard-disk drives, it’s a good idea to step back and look at the whole pricing picture. This is a confluence of the relative cost per TB of flash die versus HDD assemblies, the impact of SSD performance on server count for a given workload, and the differential in markups by OEM vendors to their end users.
The capacity of flash die has been increasing at an explosive rate over the last year. The “simple” concept of stacking flash cells in the third dimension, coupled with the stacking of these 3D die on top of each other to make a “super-die” has grown capacity by as much as 256 times per flash chip. To put this in perspective, HDD capacity took over 20 years to achieve what SSDs have done in a single year.
I believe SSDs beats HDDs in most use cases today based on total cost of ownership. I’m not just talking power savings, which are typically $10 or $12 per year. SSDs are blindingly fast and that makes jobs run fast, too. The result is you need fewer servers and in many cases these savings offset the additional costs of SSDs.
TCO calculation and the cost comparison between SSD and HDD is complicated by model class and drive markup approaches by vendors. Traditionally, we distinguished enterprise drives with dual-port SAS interfaces from nearline drives with SATA. This distinction has fallen apart in SSDs. Many storage appliances don’t need enterprise dual-port drives, while NVMe is replacing SAS and soon SATA as the SSD interface. For many applications, low-cost SSDs are adequate for the job, which changes buying patterns.
Typical OEM vendor markup ratios are as much as 14X for SSDs, making them even more expensive than raw cost would suggest compared with HDDs that typically see 10X markups or less. COTS systems are starting to drive these markups down, while buying from drive makers directly (if you are a major cloud service provider) or from master distributors (for mere mortals) opens the door to much lower SSD prices.
There are underlying trends in IT that factor into the cost of storage. First, we are rapidly migrating away from the traditional mainstay of storage, the RAID array, to more compact storage appliances that have much more software content, and, with fewer SSD drives, are able to deliver much more data. Second, the new storage appliances use the high bandwidth of SSDs or flash to compress stored data as a background job. HDDs are too slow to do this. The result is much more storage for the same price.
Let’s look more closely at these factors that make SSDs more economical in the long run.
Flash die density and pricing
Because flash memory uses very little power in operation, packing layers of storage cells in 3D is relatively straightforward. Manufacturers are either delivering or promising to stack 48, 64 and soon 96 layers on a single die.
There have been some serious technical problems along the way, but manufacturers have mostly overcome them. One issue is that it's particularly difficult to get the vertical alignment of the layers tight enough. A simple answer has surfaced: Using a coarse interconnect scheme to layer stacks of, say, 64 cells into “super-die” with as many as four super-die giving a total of 256 cells vertically.
Because the layers each need many deposition process steps, die cost savings are much smaller than capacity boosting would suggest, at maybe 2X to 4X. Currently, too, the transition to 3D NAND is running later than expected and there is a die shortage as NAND demand takes off. The resulting price blip should go away by mid-2018 and 3D NAND will cause per-terabyte prices to drop significantly.
However, there are savings in packaging and support chips around the new 3D NAND. Today, 1 TB SSDs in a tiny M2.0 footprint are sold at price parity with 2.5 inch SATA SSD drives.
(Image: Pozdeyev Vitaly/Shutterstock)
SSDs are roughly 1000X faster than HDDs on random I/O, with bandwidth anywhere from 5X to 100X compared to the fastest HDD. Numbers like this dramatically impact your workload. Historically, servers have become more and more I/O-limited as core count has increased while hard disk performance stagnated.
This, more than $/TB, is the prevailing metric in the new storage world: If an application runs twice as fast, I need half the servers to run it! That is a massive saving, which usually more than offsets the extra cost of SSDs today. Upgrading existing servers with SSDs becomes a viable alternative to a fork-lift upgrade, buying two or three years of useful life for the servers.
This also works for virtualized servers, boosting IOPS/instance by big factors and, with containers coming fast, can handle the large increase in I/O performance containers demand.
(Image: Quality Stock Arts/Shutterstock)
Unless you are buying 10,000 drives in one shot, you’ll be dealing with either an OEM or a distributor. Both must make a profit, but their typical markups are much different. Distribution typically prices at less than 2X for generic SSDs, which is close to internet pricing, though with a somewhat better handle on quality than most online vendors.
Traditional OEM vendors, on the other hand, ask for much higher markups, going as high as 14X their cost. If we get around the vendor hype of “better quality, proper certification," this is a huge number, even if discounted by 50%.
Moreover, vendor lock-in can make it hard to switch drives, however, this industry practice is thankfully changing. The use of COTS systems with open-source or third-party software has weakened the lock-in approach and some server/storage vendors, notably SuperMicro, have already abolished the practice and allow inexpensive drives from distribution to be used.
With distribution pricing for SSDs, the TCO calculation swings strongly in favor of SSDs.
“Enterprise” drives no longer required
There is a strong argument to be made that dual-ported drives offer no value in today’s enterprise systems architectures. Virtualization and automated, orchestrated recovery, on the one hand, and new data-integrity approaches such as replication, erasure-coding and appliance-level redundancy, on the other, mean that the incremental value of dual ports for data integrity is minimal.
Take dual-porting away from enterprise drives and what’s left? Maybe longer wear life, usually achieved by over-provisioning the flash space, higher performance due to a high-end flash controller, possibly coupled with better error correction schemes. What's interesting is that these features appear in nearline drives, too, while SSD vendors offer a spectrum of performance and wear life in a typical model family.
My advice is to get savvy about the spectrum of SSDs offered and fit them to your need. It will save money compared to blindly buying “enterprise” drives every time a server storage need surfaces.
NVMe, SAS, and SATA
Interfaces matter in hard drives. SAS equals “enterprise” while SATA is “nearline” or “secondary bulk” storage. NVMe has taken almost all of the SAS use cases for SSD. It’s much faster, has 3% of the SAS server overhead at much higher bandwidth than SAS can sustain. It’s also eating into the SATA market, driven by M2.0 form-factor modules for gaming PCs and servers.
My sense is that SAS will fade away; NVMe is that much better! The SATA space is more complicated. We’ll see 100 TB 2.5 inch SSDs in 2018/2019. Are these SATA or NVMe? The internal parallelism of these 100 TB class drives, which allow concurrent reads and writes to multiple die, is comparable to today’s high-end NVMe drives, so my guess is they’ll be NVMe rather than SATA, too.
It’s interesting that the PC market, which one would expect to stick with SATA, is leading the charge to support NVMe and also M2.0 ports. The implication is that vendor emphasis will move to value propositions such as long wear life and low error rate and the result will be the ability to buy more efficiently and at lower prices.
(Image source: Intel)
Smarter storage software
Most all-flash arrays offer data compression as a standard feature, either during transfer to secondary or networked storage or even on primary data itself. This feature is unique to SSD/flash systems, since it requires substantial extra I/O bandwidth. HDD-based systems can’t provide the needed performance.
Compression is a major cost consideration for many use cases. A 5X reduction in stored image size is typical, though use cases vary from as low as no improvement to over 100X. Compression software is available for server-based storage, such as in hyper-converged systems. Likely, the use of NVDIMMs will drive the need for compression to a must-have level, simply for the bandwidth reduction into NVDIMM/DRAM or across the clusters LANs.
When storage compression and the added boosts to cluster performance and latency it delivers are factored in, using SSDs becomes a no-brainer!