“I’m not dead yet.” The classic line from Monty Python and the Holy Grail is an apt description of the hard drive. Despite the low price of flash and the innovation around SSDs, in part due to NVMe (non-volatile memory express), there’s still a healthy market for the spinning disk.

“There are a lot of people who say the whole world is going to move to flash but, you know, I'm not one of them,” said Jim Handy, principal analyst of Objective Analysis. “There's still a very good place for hard drives in what they call ‘cheap and deep’ storage.” And as much as flash has come down in price to be less of a premium media, there’s still a 10-to-1 gap between NAND flash and hard disk drive prices, he noted. “That is not going to go away.”

In the meantime, the line between storage and memory continues to blur as the number of tiers in the storage hierarchy has expanded. Once it used to be as simple as DRAM at the high end and spinning disk at low end, but the flash in the middle has grown, and Intel is hoping there’s a place for Optane in there as well. Even tape remains a viable storage option for offsite backups, and there are multiple caches in today’s processors. “There's no good reason to limit the number of tiers as long as you can use more tiers to make your system perform better at a lower cost,” said Handy.

What’s changing is that hard drives are being used in a more intelligent way, thanks to software and deployment architectures — much like how SSDs are benefitting from NVMe and NVMe over Fabric. Western Digital recently announced its Zoned Storage initiative that leverages hard drives as part of a storage design aimed at cloud and hyperscale data centers. The architecture includes shingled magnetic recording (SMR) HDDs combined with the emerging Zoned Namespaces (ZNS) standard for NVMe SSDs to deliver the necessary capacity to meet zettabyte-scale storage demands as well as better endurance and predictable, low-latency QoS performance.

Western Digital’s Zoned Storage initiative groups similar data into larger chunks to organize workloads for better performance efficiencies, putting writing sequential data to SMR hard drives while also making use of advances in NVMe SSDs. (Source: Western Digital)

The use of SMR HDDs demonstrates that not all emerging workloads require NAND flash or even Optane, but still need something other than general-purpose architectures, according to Eddie Ramirez, senior director of product management at Western Digital. More of this data, such as video, internet of things (IoT), edge, and surveillance data are sequential in nature. The Zoned Storage initiative groups similar data into larger chunks to organize workloads for better performance efficiencies, including making more use of available storage on a hard drive as high as 20TB, he said, with the company’s 15TB Ultrastar DC HC620 SMR HDDs already shipping in volume.

Ramirez said Western Digital is in a unique position in that it plays in both the SSD and hard drive markets. “We're seeing the trends from both sides. There will continue to be a place for both SSD and HDD in the data center. If you look at the amount of NAND capacity in the world, it’s not going to scale with the data growth rate.” He said there always needs to be combination of both, and that spinning disk will always cost significantly lower than flash per terabyte.

What’s changed is that the “one size, one type of HDD fits all kinds” paradigm doesn’t work anymore, said Ramirez, so just as SSDs have been segmented over the past few years for different workloads, there’s a need for purpose-built HDD storage optimized for the workloads that are going to dominate the data center in the coming years. A decade ago, the data going into cloud data centers was primarily databases from ERP systems — it was very structured, he said, but today’s data is a lot different. It’s comprised of video streaming surveillance data, for example, as well as smart city device data enabled by LTE (long-term evolution). “A lot of it will be continuous. A lot of it will be large kinds of data that need to be stored very differently from the old kind of database dominated data.”

In a sense, said Ramirez, the devices themselves are the building blocks being used in the data center alone with the trends of object storage and software defined storage, which is central to the thinking of the Zoned Storage initiative. In addition, much of the data from these emerging workloads can be written sequentially, and SMR hard drives are able to write this data closer together on high capacity drives. It’s not so much that there’s been any real innovation at the actual media level; rather, there is more intelligence in the architecture — in the same way that NVMe unlocked the benefits of using flash, which had previously been deployed using HDD architectures.

Eric Burgener, research vice president within IDC's Enterprise Infrastructure Practice, said Western Digital’s Zoned Storage reflects the fact that for random workloads, flash is clearly better, or perhaps other emerging solid-state technologies like storage class memory, “but for streaming workloads disk drives actually provide very good performance.” And they’re much cheaper, especially when you can rack up a lot of them — flash is still 30 cents a gig in an enterprise-class SSD. “If you're talking about streaming workloads like video for example, or big data analytics that don't necessarily require extremely low latency and it's all streaming style workloads, you can get actually quite good performance very cost effectively from hard disk drives.”

Burgener said the new architecture that weaves hard drives together with SSDs isn’t just a response to how much data is out there, but also to a fundamental shift in terms of the types of data. The newer data associated with cloud, artificial intelligence, machine learning, and deep learning is quite different from the classic data that flash was very good at handling, which are small block, random, workloads. “What Western Digital is doing with the Zoned Storage initiative is helping to define a new driver definition that will accommodate both NVMe and SMR HDDs. There's an intelligence layer between the application side and the drive itself, that basically tells the drive, ‘hey, this is the IO profile you can expect.’ And then it'll go to the zone that's been defined with the devices that can deliver that performance at the best cost.”

Beyond the data center demand for storage, driven a great deal by the Facebooks and Googles of the world, there has been a big decline in the number of hard drive units shipped, particularly for client devices, according to Seagate principal product manager Sinan Sahin, but there’s still growth in terms of density, especially in the cloud and hyperscale segment. All the SSDs in laptops, tablets, and smartphones are accumulating data that is being backed up to the cloud and ends up on large capacity spinning disk, he said. “We're delivering 16TB on one spindle — basically one drive, the goal within the cloud and hyperscale segment is lowering the dollars per terabyte and increasing as much of the storage as possible.”

The industry is in fact having a hard time keeping up with the storage demand of data centers, according to Sahin, and there’s also plenty of opportunity in network-attached storage too for organizations large and small, including the home business, that require on site, redundant storage. Edge computing applications such as surveillance is also a big growth area of hard drives, he said.

In the meantime, there’s also innovation happening as efforts are being made to reduce the number of mechanical parts in hard drives that in turn reduce the potential failure points to help compete with the reliability of flash. There are also more efficient methods of writing data, such as microwave-assisted magnetic recording (MAMR) technology for ultra-capacity HDDs and heat-assisted magnetic recording (HAMR). While the latter involves new materials sciences, MAMR uses a spin torque oscillator to generate a microwave field that increases the ability to record data at ultra-high density without any negative impact on reliability.

Sahin said ultimately the growth rate of data and the redundancy required for its safekeeping is what will keep hard drives — and the need to innovate — relevant. As data grows and customers get more creative about using it, the challenge is how to deliver the next layer of storage, and the next, he said. “There’s that thirst that we’ve got to feed, and as we feed that, there’s even more.”

By: DocMemory
Copyright © 2023 CST, Inc. All Rights Reserved