What is Fibre Channel over Ethernet (FCoE)? How it works, benefits, challenges and use cases

Purpose of Fibre Channel over Ethernet

Traditionally, organizations have used Ethernet for TCP/IP networks, such as local area networks (LANs), and Fibre Channel for storage area networks (SANs). Fibre Channel protocol supports high-speed data connections among computing devices that interconnect servers with shared storage resources and between storage controllers and drives using a SAN.

FCoE shares Fibre Channel and Ethernet traffic on the same physical cable. It also lets organizations separate FC and Ethernet traffic on the same hardware and helps to consolidate input/output (I/O), reduce switch complexity and cut back on cable and interface card counts.

FCoE is often compared to Internet Small Computer System Interface (iSCSI), an IP-based storage networking standard; however, they are different standards. FCoE is not designed to replace iSCSI but to facilitate the creation of a converged, interconnected fabric of network devices, logical connections and physical cables.

How Fibre Channel over Ethernet works

FCoE uses a lossless Ethernet fabric and its own FCoE frame format. It retains Fibre Channel device communications but substitutes high-speed Ethernet links for FC links between devices.

FCoE works with standard Ethernet cards, cables and switches to handle FC traffic at the data link layer of the Open Systems Interconnection model. It uses Ethernet frames to encapsulate, route and transport FC frames across an Ethernet network using Ethernet ports from one switch with FC ports and attached devices to another, similarly equipped switch.

By encapsulating FC frames into Ethernet frames, FCoE enables FC frames to run alongside IP frames. In other words, FC traffic can run over the Ethernet infrastructure. During the encapsulation process, FCoE uses one-to-one mapping so the FC frames are never segmented. Also, multiple FC frames are never put in a single Ethernet frame. This is what lets the FCoE protocol take FC traffic and put it over high-speed Ethernet infrastructure.

To make this work, a special type of network adapter called a converged network adapter (CNA) is used. A CNA combines the functionality of a Fibre Channel host bus adapter (HBA) with that of an Ethernet network adapter. The CNA not only provides the required physical connectivity but also enables lossless Ethernet. This is essential because FC is a lossless protocol and SANs expect lossless communications.

CNAs can be hardware- or software-based. In hardware-based CNAs, physical hardware manages the FC stack and FCoE processes. Software manages these aspects in a software-based CNA.

Who uses FCoE?

Fibre Channel typically appears in data centers where block storage devices are connected using fiber optic cables, owing largely to their bandwidth. Fiber optic cables have a distance limitation, which can be addressed when Ethernet is used.

Assuming the SAN and associated storage devices are situated within a data center, FC can be used effectively. If storage devices are located outside a data center -- for example, at another campus location or elsewhere in the building -- FCoE is a good alternative because it can handle greater distances without any degradation in bandwidth.

Organizations that need to connect storage devices but wish to keep costs down might opt for FCoE because the cost for a twisted pair cable for Ethernet is lower than that of a fiber optic cable -- not counting costs for FC switches and connecting devices. It's also easier for twisted-pair cable to go around corners, whereas fiber optic cable must observe specific radius limits when traversing corners or other cable routes.

Benefits of Fibre Channel over Ethernet

There are a number of benefits to using FCoE, including the following:

• Reduced costs and complexity. By facilitating FC communications over conventional Ethernet networks, FCoE eliminates the need for specialized and expensive Fibre Channel switches and other hardware, such as additional cabling. This reduces the networking costs as well as some aspects of network complexity.
• Increased performance. Fibre Channel is a high-speed storage protocol, with some storage devices supporting speeds of up to 128 Gbps. Even so, such devices tend to be expensive. While some organizations might still use 8 Gbps storage devices and SANs, FCoE makes it possible to achieve speeds up to 100 Gbps by using comparatively inexpensive Ethernet networks.
• Simplified storage management. Many tools exist for managing and monitoring Ethernet networks. Because FCoE routes storage traffic across a standard Ethernet network, an organization can use its existing networking tools to manage storage data traffic.
• Lower power consumption. FCoE's ability to consolidate LANs and SANs helps lower power consumption within data centers. FCoE also improves data center efficiency and simplifies its management.
• Support for high-performance storage applications. FCoE enables organizations to combine Ethernet speed with Fiber Channel reliable and lossless data transport capabilities, high performance, low latency and enhanced security.
• Cloud environment suitability. FCoE can provide high-performance, low-latency storage connectivity over Ethernet in cloud environments. These environments are both scalable and more cost-effective than pure Ethernet or pure Fiber Channel SANs.
• Support for quality of service (QoS) goals. Like native Fiber Channel, FCoE ensures reliable and high-performance data transfers. It also guarantees required bandwidth for high-priority storage traffic, supporting QoS.

Challenges of Fibre Channel over Ethernet

FCoE is not without its complexities, and organizations might encounter the following challenges when using FCoE:

• Network latency. FCoE relies on Ethernet for data transport, which introduces additional network latency compared to traditional FC connectivity. Ethernet networks can have inherent latency due to factors such as packet processing, congestion and network equipment. Latency can cause performance and responsiveness issues.
• Possible complexity from convergence. FCoE enables the convergence of storage and data networking over a shared Ethernet infrastructure. This convergence can introduce complexity in terms of network design, configuration, management and troubleshooting.
• Compatibility and interoperability. FCoE requires compatibility and interoperability between different vendors' products, including Ethernet switches, FC switches, HBAs and storage arrays. Ensuring seamless integration and compatibility among various components can be a challenge, particularly when dealing with older or heterogeneous infrastructure.

FCoE use cases

The following are examples of how FCoE can provide value to a storage infrastructure:

• High-performance storage. Applications that require high-performance, low-latency and lossless storage can benefit from FCoE. These include applications that have large-scale data storage requirements, use high-speed online transactional processing databases or require frequent, rapid or real-time data updates.
• I/O consolidation. FCoE allows for I/O consolidation on a unified network for all traffic without requiring new hardware or a total rip-and-replace of the existing infrastructure. This not only reduces costs but also simplifies storage management. Additionally, FCoE is suitable in data centers where space saving, power saving and the need for less cooling are key requirements.
• Legacy hardware. Organizations can use FCoE to introduce an Ethernet-based SAN into the FC-based data center in order to improve storage performance -- without having to invest in expensive modern FC hardware or disrupting existing administration workflows.

FCoE vs. other storage protocols

There are some similarities between FCoE and iSCSI. However, there are differences as well.

One major difference is that FCoE encapsulates native Fibre Channel packets inside Ethernet packets. However, iSCSI encapsulates native SCSI commands inside IP packets.

Another difference is that FCoE is designed to enable Fibre Channel communications across a high-speed Ethernet network and is generally used within the data center. On the other hand, iSCSI can be used for communications among devices within a data center and can also be used as a means of connecting to a remote storage device. Additionally, FCoE requires a converged network adapter, while iSCSI can be used with any standard network interface card.

Finally, SCSI environments are usually small configurations with a few servers. Fibre Channel environments tend to be much bigger, usually containing hundreds or thousands of nodes. FCoE can be used for such environments to provide Fiber Channel over lossless Ethernet.

The future of storage is in collaboration and unified management capabilities. Find out what to look for with your next storage investment.