SAS vs. NVMe: The future of the two key storage interfaces

SAS assessment

SAS is part of the ongoing SCSI development effort led by the T10 Technical Committee of the InterNational Committee on Information Technology Standards and backed by more than 50 companies.

Performance and scalability

When you look at the spec sheets, SAS drives inevitably come up short in the SAS vs. NVMe PCIe SSD comparison. However, there's more to it than the spec sheet numbers, according to Brett. For instance, in a lane-by-lane comparison, a 24G SAS link with 2.4 GBps throughput outperforms a single PCIe 4.0 lane at 2 GBps, Brett said.

The pendulum has swung; there's no looking back. NVMe is now the dominant interface for all enterprise SSDs.

NVMe PCIe SSDs typically use four lanes to connect to a host, which means they end up using more host resources, such as memory and power, than SAS drives, which typically use one or two links, Brett said. In addition, most applications don't need the million IOPS provided per NVMe drive, he added, but those drives use more host resources to service that level of performance.

SAS also comes up short in the SAS vs. NVMe comparison in terms of scalability. But here, too, there are extenuating circumstances, Brett said. "SAS may not scale at the same rate as NVMe, but [it] scales in a different way," he said.

SAS is designed to accommodate configurations of 64,000 end devices. Given the amount of memory needed, that many devices aren't usually supported, Brett said. However, setups of around 2,000 devices can be found in real-world configurations.

In addition, it's common to run SAS-based arrays connected to a SAN that's built on SCSI technology. SANs typically have multiple hosts, Brett noted, and you can dynamically scale SAS storage by adding disks or arrays without shutting down the system. SAS enables cross-rack, -room and -data center connectivity options, he added.

Flexibility

SAS, with its 2.5- and 3.5-inch drive form factors, has fewer form factor options than NVMe SSDs. But, Brett noted, there are other ways SAS demonstrates flexibility. SAS connects to more device types than NVMe, and it has more flexibility at the architectural level, he said.

SAS connections and backplanes natively support SAS and SATA protocols, hard drives and SSDs, as well as optical and tape drives. "That's a few more device types on a SAS connection than NVMe does today," he said.

SAS also lets you create fully tiered environments with a caching layer; a warm tier that houses large amounts of regularly accessed data; and an archival cold tier that uses low-cost, high-capacity hard drives, Brett added.

SAS manageability and serviceability

The SCSI Enclosure Services (SES) protocol was created as part of the SAS-3 spec, providing in-band and out-of-band ways to communicate with a SCSI array and the devices inside an array. The enclosure houses SCSI devices and non-SCSI supporting devices, such as power supplies, fans, displays and sensors, Brett said. A controller manages the connection, and management tools are available that work with and support the SES.

NVMe assessment

Prior to the development of the NVMe spec, PCIe SSDs were based on proprietary technology. NVM Express Inc. was formed to develop an open industry standard to take advantage of the high-performance, low-latency capabilities of solid-state technology.

"If it's a protocol designed for SSDs, you don't really need all the legacy stuff that was designed for hard drives. You can take all that out," Hands said.

Performance and scalability

Among the features that enhance NVMe performance is the queue depth, Hands said. SAS supports one queue with a 256-queue depth. In contrast, NVMe was designed to support 65,000 queues, each having a queue depth of 65,000. After the overhead, an optimized PCIe Gen 4x4 drive reach speeds of 7.2 to 7.4 GBps -- about 23 times faster than a SATA drive and six times faster than a SAS 12 GBps SSD, he said.

NVMe "was designed to be scalable, and this is really important for steps going forward for virtualization and containerized architecture and cloud-based storage," he noted.

On the latency side, because NVMe drives are attached directly to the CPU via PCIe, you get a latency reduction of 15 to 25 microseconds. That's a significant improvement for NAND SSDs, and it will be important for storage class memory drives that currently have average latency of six to eight microseconds, Hands said.

NVMe also doesn't require uncacheable or memory-mapped I/O register reads in the command commission or completion path, and a maximum of one MMIO register write is necessary in the command commission path, he added. This significantly reduces the latency overhead and will make it possible for new innovative drivers to take advantage of enhanced performance.

Flexibility

NVMe SSDs come in a variety of form factors, reflecting the ubiquitous nature of the PCIe interface. The three main types are:

• AIC/CEM (add-in card/card electromechanical) is the general-purpose form factor for PCIe in different sizes and heights that most SSDs use;
• M.2, developed for mobile devices and laptops, is also used in data centers and by hyperscalers; and
• U.2 is the 2.5-inch form factor that makes up the majority of NVMe SSDs today.

With PCIe Gen 4 SSDs, NVMe is starting to transition to Enterprise and Data Center SSD Form Factor (EDSFF) purpose-built storage form factors for data centers. E1 is designed for 1U, with versions being developed for high-capacity drives, scalable performance and mainstream compute use. E3, in various lengths and heights, is optimized for 2U compute and storage.

Another flexibility plus for NVMe vs. SAS, Hands said, is that because PCIe is so ubiquitous, it offers a variety of attach points, including the CPU, PCIe switches and the PCIe retimer for link extension or port expansion.

Manageability and serviceability

NVMe developers didn't see any reason to reinvent the wheel when it came to manageability, Hands said. The NVMe management interface -- the NVMe-MI -- was built upon the SES enclosure management capabilities. NVMe-MI provides in-band and out-of-band management of NVMe devices and one management console that supports multiple OSes.

The most common way people manage NVMe devices is using in-band management to send commands directly through the admin queue to the drive, according to Hands. "The official management capability for NVMe is just built into the operating system," he said. "If you install NVMe [command-line interface] in Linux, you can basically run any single command you can possibly think of in NVM Express that's in the spec through the command line. It's all just built into the tool."

SAS vs. NVMe conclusion

When it comes to performance and scalability at the device level, NVMe outperforms and out scales SAS, Jeanette said. However, when looking at SAS vs. NVMe scalability outside the box, it's a different story. "As of today, SAS may have the benefit here," he said.

More is better when it comes to form factors, Jeanette concluded. NVMe PCIe's many form factor choices outdo the limited SAS options, but choice comes with risks, he noted. In this case, there's the possibility that NVMe design choices made today might not be available or supported in the future.

"With SAS, you know the form factors," Jeanette said. "You know what you have today and what you're going have years from now. However, the design options might lag vs. your PCIe options."

With manageability, NVMe-MI had the advantage of decades of work developing the SES protocol, which provided the building blocks for the NVMe management spec. However, NVMe-MI also takes manageability and serviceability to the next level, Jeanette said.

Bottom line in the SAS vs. NVMe comparison: SAS is a tried-and-true technology that will continue to play a significant role in the enterprise and won't go away anytime soon, according to Jeanette. While NVMe PCIe drives are being used to a large extent by hyperscalers and, to a lesser extent, enterprises, he said, it's on track to be the storage interface of the future.