https://www.techtarget.com/searchstorage/feature/NVMe-oF-products-are-changing-the-face-of-data-storage
The NVMe interface is rapidly becoming the preferred interconnect for flash disks and all-flash arrays because of its superior performance and lower system overhead. Indeed, even as IDC recently warned of a sales slowdown for external storage in the EMEA region, it noted that the AFA market was red hot -- up 20% annually to 35% market share. Storage buyers are clearly ready to spend precious capital on better performing NVMe devices rather than SSDs using legacy storage interfaces.
NVMe has begun displacing SSDs in servers. However, with organizations using clusters of VMs and container servers for most workloads, it hampers workload portability if storage is locked down to a particular system. Fortunately, NVMe-oF has finally made it through the specifications process and NVMe-oF products are starting to provide a viable option for NVMe-based shared storage.
A fundamental feature of NVMe-oF is binding support, namely the ability to operate over various underlying transport fabrics including Fibre Channel, InfiniBand and Ethernet. Protocol bindings serve as the connection between NVMe and the network transport, but due to the technical differences of various network protocols, binding also places restrictions on NVMe capabilities and defines how NVMe is managed using the underlying fabric.
We gave advice on evaluating NVMe-oF options in this article. Although the specification for NVMe-oF over TCP lagged the other bindings, it was finally ratified in late 2018 and ushered in an era in which NVMe-oF hosts and controllers can communicate over any standard IP network. While the TCP specification focuses on software implementations using the TCP stacks on host OSes and AFAs, it doesn't preclude hardware-accelerated implementations.
As a young technology, the ecosystem of NVMe-oF products is rapidly evolving. However, it has now reached a degree of maturity, making NVMe-oF devices suitable for production workloads. The following is a look at some of NVMe-oF product categories, devices, software and supported systems.
Each week, the tech news wires are full of NVMe product announcements, many of which won't be released for months. The hype makes it hard to keep track of the market, much less figure out which products are compatible with one another.
Fortunately for technology buyers, the University of New Hampshire InterOperability Laboratory (UNH-IOL) has long served as a clearinghouse for reliable information on network devices and their support of various network standards. UNH-IOL has been testing NVMe-oF products for two years and its integrators list is an excellent resource for finding standards-compliant products.
The list includes several product types, such as:
Compliance testing is a tedious process that's dependent upon vendors submitting products they believe will pass. Hence, it necessarily lags product introductions, particularly in a market as dynamic as NVMe.
The following is a sample of some of the storage systems supporting NVMe-oF:
1. Separate control and data plane hardware with x86 servers running the storage software and data spread across NVMe-oF disk enclosures.
2. A high-speed Ethernet or InfiniBand network fabric connecting compute and storage nodes. Compute nodes are typically housed in 2U quad-server enclosures with four 100-GbE NICs, while the storage nodes are also a 2U chassis with 44 x 15.36 TB quad-level cell flash disks and 12 x 1.5 TB U.2 XPoint devices.
3. Distributing metadata across the storage nodes on dedicated fast Intel Optane 3D XPoint memory in the NVMe-oF enclosures. Such a distributed metadata design enables the compute nodes to be stateless and use non-redundant, lower-cost systems.
As the UNH-IOL test data indicates, most major Linux OSes, including RHEL, CoreOS, SUSE and Ubuntu, provide NVMe support, as do all recent versions of Windows Server (2012, 2016) and the Microsoft Windows 10 client.
NVMe-oF is a critical foundation for composable infrastructure that enables physical hardware components -- servers, storage capacity and network interfaces -- to be logically carved up into virtual instances that are allocated to particular VMs, container clusters and application servers. Aside from being the foundation for next-generation cloud data centers, NVMe-oF products are particularly attractive for analytics, AI training and high-performance computing applications with high levels of storage IO that can take advantage of RDMA to significantly reduce IO overhead on compute servers.
25 Sep 2019