Storage options for virtual machines: Raw device mappings, VMFS

to use VMFS, the proprietary, high-performance clustered file system provided with VMware Infrastructure (VI). Using virtual disks (VMDK files) on VMFS is the preferred option for most enterprise applications, and as such supports the full range of functionality available in a VI implementation, including VM snapshots, VMotion, Storage VMotion, and VMware Consolidated Backup (VCB).

The other way to provision storage is Raw Device Mapping (RDM). RDMs are sometimes needed in instances where virtualized access to the underlying storage would interfere in the operation of software running within the VM. One such example is SAN management software, which typically requires direct access to the underlying hardware; and thus would need to use an RDM instead of a virtual disk. In this tip, I'll discuss what RDMs are and when to use them over a virtual disk.

Defining raw device mappings
An RDM is a file that resides within a VMFS volume that acts as a proxy, or an intermediary, for a raw physical device. One can think of an RDM as a symbolic link to a raw LUN. The RDM contains metadata and other information about the raw physical device being accessed and can, depending upon the configuration of the RDM, add features like VMotion support and snapshots to VMs that are using raw LUNs.

Why use RDMs instead of virtual disk files? VMware positions VMFS as the preferred storage option for enterprise applications, including file servers, web servers, databases, ERP, and CRM. In previous releases of ESX Server and VMFS, RDMs were often used to minimize VMFS overhead in an attempt to improve disk performance. Based on comparisons done by VMware with VMFS-3, however, disk performance between VMFS-3 and RDMs is very comparable in most instances. Refer to Performance Characterization of VMFS and RDM Using a SAN for more information and details on the performance comparison between VMFS and RDM with ESX Server 3.5.

If performance is comparable between VMFS and RDM in most instances, then what other reasons might there be for using RDM instead of VMFS? Usually, the answer boils down to application requirements. SAN management software has already been mentioned as one application that needs RDMs instead of virtual disks; this is due to the direct communication between the SAN management software and the storage array. There are other instances as well in which RDMs are needed instead of virtual disks:

• RDMs are necessary in MSCS clusters for the quorum and data disks. This includes virtual-to-virtual clusters across ESX hosts as well as physical-to-virtual clusters. Note that virtual disks can be used for "cluster-in-a-box" configurations on a single physical host.

   

• RDMs are necessary in situations where SAN-aware applications are running inside a VM. There are many examples of this; one such example is NetApp's SnapManager series of applications. These applications require direct communication with the storage array and therefore cannot use virtual disks.

   

• RDMs must be used in situations where N_Port ID Virtualization (NPIV) will be used. NPIV allows a single Fibre Channel HBA port to register with a Fibre Channel fabric using multiple Worldwide Port Names (WWPNs). NPIV support is new to ESX 3.5 and allows ESX to present a "virtual HBA" to the VM. NPIV can be used only with RDMs.

RDM limitations
There are two types of RDMs: virtual compatibility mode RDMs and physical compatibility mode RDMs. Physical mode RDMs, in particular, have some fairly significant limitations:

• No VMware snapshots
• No VCB support, because VCB requires VMware snapshots
• No cloning VMs that use physical mode RDMs
• No converting VMs that use physical mode RDMs into templates
• No migrating VMs with physical mode RDMs if the migration involves copying the disk
• No VMotion with physical mode RDMs

Virtual mode RDMs address some of these issues, allowing raw LUNs to be treated very much like virtual disks and enabling functionality like VMotion, snapshotting, and cloning. Virtual mode RDMs are acceptable in most cases where RDMs are required. For example, virtual mode RDMs can be used in virtual-to-virtual cluster across physical hosts. Note that physical-to-virtual clusters across boxes, though, require physical mode RDMs.

While virtual disks will work for the large majority of applications and workloads in a VI environment, the use of RDMs--either virtual mode RDMs or physical mode RDMs--can help eliminate potential compatibility issues or allow applications to run virtualized without any loss of functionality.

ABOUT THE AUTHOR: Scott Lowe began working professionally in the technology field in 1994 and has since held the roles of an instructor, technical trainer, server/network administrator, systems engineer, IT manager, and CTO. For the last few years, Scott has worked as a senior systems engineer with a reseller, providing technology solutions to enterprise customers.