Columnists

CES: Unclear prospects for hybrid, solid-state hard disks

Is the time finally right for flash-augmented hard-disk drives? What about fully solid-state storage?

By Brian Dipert, Senior Technical Editor -- EDN, 1/8/2007 7:03:00 AM

Several days before Christmas, I took a briefing from representatives of several of the five founding companies (Fujitsu, Hitachi, Samsung, Seagate and Toshiba) that currently comprise the Hybrid Storage Alliance. Although the news of the group's formation broke on Thursday (PDF), I decided to wait until today to comment on this latest hybrid-HDD announcement, so that I could incorporate the output from a "Flash and HDD's, Where's the Cache?" panel session held late Sunday morning at the Storage Visions Conference. I am also now able to incorporate the perspectives of several Toshiba representatives, with whom I had breakfast Sunday morning.

Flash-memory-inclusive HDDs aren't anything new (I've covered them extensively in my blog, Brian's Brain; this chronologically ordered list of references will get you up to speed).

The fundamental motivations behind the proposal to supplement a HDD with a small amount of flash-memory cache (256 Mbytes on average, but potentially extending below that amount to address cost concerns, or above it for added systemic effect) are multifold. First, the schemes promise faster performance, especially for reads (both system boot and normal operation). Simultaneously, hybrid approaches reduce power consumption by keeping the HDD in its deep-sleep modes a higher percentage of the time. All this is enabled by flash memory's nonvolatility, versus the SRAM and DRAM that currently (and will probably continue to) act as HDD caches both in drives and in systems.

Two competing proposals exist to address the potential market need for a flash-augmented hard drive. Intel's Robson Technology places a flash-memory chip or array in the system, either directly on the motherboard or on a mini-sized PCI Express module. The system CPU, in conjunction with Intel-supplied drivers, manages the system.

The alternative approach advocated by the Hybrid Storage Alliance involves placing the flash memory inside the HDD, with an on-drive media-controller IC orchestrating affairs. Both approaches require operating-system awareness of the flash-memory presence; while "hooks" for both approaches are built into Windows Vista, for example, Microsoft does not currently plan a retrofit for Windows XP.

The crux of the Hybrid Storage Alliance's advocacy of on-drive flash memory versus Robson is captured in this PowerPoint slide (JPEG) from the presentation the group shared with me two weeks back. Many of the issues are overemphasized for effect. For example, the supposed "large negative system chassis impact" of Robson will likely provoke a chuckle from anyone who's ever encountered a µBGA-packaged or encapsulated bare-die flash memory, or a miniscule PCI Express Mini Card. Several of the points in the Hybrid Storage Alliance's pitch, however, bear closer focus.

Data Integrity: Alliance representatives painted a gloomy scenario of a HDD somehow getting removed from a powered-up system, thereby creating an out-of sequence disparity between the HDD contents and any system-written data in the Robson cache that hadn't yet been mirrored to the hard drive. Putting aside for a moment the unlikelihood of such a scenario, as well as the user's expectations upon encountering such a scenario (would you yank a drive out of a running, or even standby-mode, system and still expect no errors when you reinserted it?), Robson doesn't materially worsen a situation that's existed since the days of MS-DOS 6.0.

SMARTDrive (smartdrv.exe) originally got loaded as a separate executable via Autoexec.bat but is now deeply embedded within the operating system. It uses a portion of system DRAM as a read/write cache for the slower HDD behind it and, by default, it caches writes to local drives in a write-back (alternatively stated, "write-behind") manner. This means that for higher apparent performance, SMARTDrive intercepts operating-system write requests, moves the data to fast-writing DRAM and returns control back to the operating system before it finishes copying the data to the HDD. Any HDD removal prior to completion of the SMARTDrive transfer would cause exactly the same doomsday scenario that the hybrid HDD advocates paint for a Robson-augmented configuration.

Second sources: As currently proposed, Robson is uniquely tied to Intel chip sets (and, probably by default, to Intel NAND-flash memory). Microsoft, which has already been working with Samsung in particular for several years and is firmly behind the Hybrid HDD effort, isn't opposed to the Robson approach. However, Microsoft's James Bordon made it very clear Sunday morning that any equally strong Microsoft sanction of a system-based flash-memory cache would be contingent on its being an "open standard on which innovation can still occur." As I mentioned last September, "bundling" arrangements of Intel CPUs, chip sets, and Robson-targeted flash memory will be fiscally tempting to PC OEMs. However, AMD-based designs will be shut out until Robson becomes more industry-encompassing, opening wide the door for the hybrid-HDD alternative.

Three of the five members of the Hybrid Storage Alliance (Fujitsu, Samsung, and Seagate) have to-date formally announced their intentions to ship hybrid HDDs. It's a fair guess that the other two will shortly follow, although as you'll soon see, nothing's guaranteed.

But where's Western Digital? Hybrid Storage Alliance members insisted, both at the late December briefing and at breakfast Sunday morning, that the group approached all 1.8- and 2.5-in. HDD suppliers as potential alliance partners. WD hasn't yet responded to my inquiry as to whether this is true and, if so, why it declined the invitation. But the company's absence casts a bit of a pall on the alliance's efforts. A lack of hybrid HDD suppliers, panelists admitted Sunday morning, was the root cause of Microsoft's recent decision to delay "Premium" logo certification for hybrid HDD-inclusive systems until June 2008.

At breakfast Sunday, Toshiba's Maciek Brzeski was refreshingly candid about his company's forecasts for hybrid HDDs. He pointed out that the HDD industry's to-date success had come from shipping extremely high volumes of a very small number of product versions, and that adding flash-augmented drives to that mix (in multiple density variants, to boot) ran counter to that efficiency trend. Assuming this interview contains an accurate quote, apparently Apple agrees with him.

In Sunday's Storage Visions panel, analyst Joe Unsworth echoed Brzeski's observation, pointing out that not only was the incremental price tag of a flash-augmented HDD unpalatable to PC OEMs, they want design flexibility, and they don't want to have to qualify a large number of additional flash-inclusive HDD proliferations from various suppliers. In contrast, Unsworth said that an optionally populated, Robson-targeted PCI Express slot with one or two Mini Card-based density options from Intel was compelling pitch. To that point, however, Brzeski pointed out that the first iteration of Intel's Santa Rosa platform will only support a maximum of two PCI Express ports; populating one of them with a Robson module might unduly restrict OEMs' flexibility with respect to other desired subsystems.

Other observations

Hybrid HDDs have at least one clear advantage over system-based flash caches—one that will be particularly appealing to the majority of EDN's readers, who are not PC designers. Because the flash memory in this configuration is contained within the HDD housing and is managed by the HDD controller, any operating system that incorporates support for the necessary ATA command set extensions is a hybrid HDD candidate.

In contrast, system-based flash memory arrays have a compelling advantage of their own. Such a cache is conceptually able to manage all storage devices connected to the system, both local (optical discs, HDDs, and externally tethered units) and remote (network-attached storage). The Microsoft representatives I talked with Sunday were unwilling to provide confirmation that such an expansion of the Robson vision is in the works for a future Vista service pack or other patch, but they did indicate that it was under evaluation.

Regardless of where the flash memory resides, which type should find use? NAND is the obvious candidate, by virtue of its fast write speeds, acceptable read speeds, and plethora of suppliers. Implementers will be sorely tempted to go with MLC (multilevel-cell) NAND to save cost but, especially in the small densities that find use in the cache application, they should give pause and seriously consider the SLC (single-level cell) alternative. Not only does it have higher read and write performance, it's also much more reliable, requiring significantly less ECC overhead (both in bits and in processing speed) to deliver the necessary system-lifetime targets.

Samsung's recent unveiling of a 16-Gbit NAND device built on 50-nm process technology symbolizes that it's possible to construct a single-chip flash cache, assuming that chip's data-bus width won't be a performance bottleneck. However, consider the significant density disparity between the flash cache and the HDD behind it. Depending on a particular computer owner's usage patterns in conjunction with the operating system's aggressiveness in harnessing ReadyBoost- and ReadyDrive-like techniques, the flash cache could experience heavy write traffic. Ensuring adequate write- and erase-wear-leveling of that flash cache will be challenging for even the most sophisticated media-management algorithm.

The solid-state alternative

Flash-memory capacity discussions provide a logical lead-in to the topic of a fully flash-based HDD—the SSD (solid-state drive). Such products have been available for many years from several small suppliers. They're useful when harsh environmental conditions or other factors preclude the use of rotating magnetic storage but the system design requires a conventional PATA or SATA interface (that is, when the system can't support a PCMCIA, CompactFlash, or other solid-state alternative). However, with flash-memory densities and manufacturing capacities both spiraling upward, and prices falling, larger companies are beginning to eye the SSD opportunity.

Last May, I mentioned that Samsung had developed SDDs up to 32 Gbytes and was beginning to offer them as options in its UMPCs and subcompact notebooks. And just a few days ago, SanDisk tossed a 32-Gbyte SSD of its own into the ring, by virtue of its earlier acquisition of M-Systems. While I appreciate SanDisk's enthusiasm in "targeting hard disk replacement in notebook computers," I'm not going to hold my breath on the company's vision coming to fruition any time soon.

Why? Review this blog post, where I compare the capacity independent, fixed bill-of-materials cost of a HDD against the more linear, capacity-versus-price trend of a flash memory-based alternative storage device. The example pricing data is nearly a year old at this point, but the flash-versus-HDD threshold hasn't moved to the point where a 32-Gbyte SSD will be a mass-market item.

How do I know? Back in May of last year I pointed out that an SSD-based laptop or tablet PC would be around $900 more expensive than its HDD-based (and likely much higher-capacity) sibling. And buried in SanDisk's press release of three days ago is the following statement:

It is projected that inclusion of the SanDisk 32GB SSD in a notebook PC could increase the end-user price by around $600 in the first half of 2007.

My request for more specific pricing information from SanDisk has, to date, gone unanswered beyond a "we'll get back to you" promise (which I honestly doubt will ever be kept). Granted, SSDs do boost system performance. Check out, for example, this YouTube-hosted video (thanks to Digg for the heads-up) of a Fujitsu demonstration of conventional HDD- and SSD-based system boot times. Gizmodo also found and commented on the video, and I disagree with the editor's "disappointed" verdict; while it's true that the SSD-based system was "only about 6 seconds faster," that time disparity represents acceleration of nearly 20%. And the lack of rotating-media motors in SSDs will also improve system battery life.

However, a multithousand-dollar laptop, which might be better able to absorb the incremental price of a SSD, will demand more capacity than Samsung and SanDisk are currently able to supply. And a system with more moderate storage needs would have its price tag unacceptably inflated by the SSD inclusion. Until flash memory gets significantly cheaper, and until users' storage density demands taper off, SSDs will be nothing more than an intellectually interesting niche product.

References

• Hybrid hard drives (2005 WinHEC)
• Flash cache (fall 2005 IDF Taiwan)
• 'Flash'y HDDs: Has their time finally come? (2006 WinHEC)
• Turf wars (fall 2006 IDF)