PCIe Flash Market Heats Up

Apparently, people who are able to put their money where their mouths are think that the market for PCIe attached flash memory is bigger than Fusion IO selling a couple of thousand cards to Facebook to speed up drunken frat-boy photo load times. Just this week, Texas Memory Systems and Micron released new high-end PCIe flash cards that push the performance and capacity envelopes for PCIe flash.

Texas Memory’s RamSan-70, code-named Gorilla for its 900 GBytes of single-level cell (SLC) capacity, uses Toshiba's 32 NAND chips to deliver about 330,000 random read input/output operations per second (IOPS) at about 2-GBps bandwidth. To manage that 900 GBytes of flash, TMS has built its own flash controller using a PowerPC CPU and field-programmable gate arrays (FPGAs). The on-board processor does the heavy lifting of write leveling and on-board RAIN (redundant array of independent NAND) like data protection.

Micron P320h, while it doesn't quite match up to the RamSan-70 on a capacity basis, delivers even more impressive read I/O performance. Micron says the little devil can deliver 750,000 random read IOPs, which is darned impressive. TMS catches up on the write IOPs, however, as the RamSan-70 delivers 400,000 and Micron can only manage 341,000. Like the RamSAN, it implements 7+1 RAIN for data protection. I don't know whether to be impressed with the read performance of the Micron or upset at how asymmetrical its performance is.

The new cards from Micron and TMS both compete not only with Fusion-io's ioDrive, but also with LSI’s WarpDrive, Virident TachION and with the PCIe flash card to come as part of EMC’s Lightning, which is also expected to hit the market as an Intel OEM or whitebox product. It seems to me that’s a lot of players chasing the high-end users that want to avoid the few microseconds of additional latency involved in crossing a SAS or SATA interface or Fibre Channel network.

Micron’s entry to the market is also significant because of its vertical integration. Where TMS, LSI, Virident and Marvell have to buy NAND chips from Toshiba or Samsung, Micron produces its own NAND in a joint venture with Intel. This puts Micron and Intel, when they enter the market, in a position to underprice Fusion-io and others, and still make a profit.My problem with PCIe flash is that most of our applications just aren’t designed to handle it. If you’re Facebook, you can write your code to run on the best platform, but if you’re in the heart of corporate IT, you build platforms to support the applications, not the other way around. So you’ll need some sort of translation layer if your whole data set doesn’t fit on the flash card.

To get the most out of PCIe flash, we need a translation layer between our standard operating systems and applications and the flash card. So far the best shot seems to be something like EMC’s Lightning, which uses the PCIe flash as a write-through disk cache. Since a write-through cache always keeps the back-end SAN data up to date, features that rely on shared storage will still work. Of course, the flash is local to the host holding the PCIe card, so a vMotion from one host to another would cause an increase in storage latency until the destination host populated its cache from the back-end storage.

Marvell’s Dragonfly PCIe flash card is designed to work with Marvell’s caching drivers to build a complete caching solution. IO Turbine’s Accelio is more interesting because it’s hardware independent, using any flash storage as cache for vSphere servers.

I’m just waiting for a solution in my price range. I don’t have any apps that need 300,000 or even 10,000 IOPs. I’m looking for the SME solution that uses $1,000 to $1,500 of flash as either a low-end PCIe flash card, like those from OCZ or prosumer SSDs like Intel’s, so I can replace the 20 to 50 15K RPM drives my clients need for their IO-intensive apps with six 2-TByte drives for capacity and still get five 6,000 IOPs. Since several vendors, including Marvell, nVelo and Intel, have flash caching solutions for desktops and enterprise solutions hitting the streets now, I hope I don’t have wait long for someone to meet me in the middle.

I hope sometime soon to see a vSphere solution that can combine local and shared SSDs with the smarts to do things like write snapshots to flash and populate the shared flash with cache data as vMotion moves loads from one host to another. Flash, trash and application-aware storage systems--now that will be cool.