While 10 Gigabit Ethernet (10 GbE) offers substantially greater bandwidth than its Gigabit Ethernet predecessor, many IT shops won't be able to make a solid case for it just yet. At its current price, 10 GbE is expensive enough to require a discussion around cost justification. In many cases, cheaper Gigabit Ethernet technology may still be adequate to meet the needs of a company's block-based iSCSI storage or file-based NAS storage.
Two of the more common 10 GbE drivers are the desire to run more virtual machines (VMs) per server and to consolidate data centers. Large hosting companies and scale-out NAS users also may see a need for the extra bandwidth for Web 2.0 or video-streaming applications.
But IT shops need to decide if any requirements are pressing enough to cause them to install 10 GbE now or to wait until prices drop further. Costs have fallen dramatically since the technology emerged more than six years ago, when a per-port price of $40,000 was no exaggeration. More recently, a range of $500 per switch port is more common, although Gigabit Ethernet is considerably less at under $100.
The average price paid to manufacturers for a 10 Gigabit Ethernet standalone adapter was $597 in the first quarter vs. $814 during the same period a year ago, according to Dell'Oro Group Inc., a Redwood City, Calif.-based market research company that tracks the networking and telecommunications industries. A Gigabit Ethernet copper standalone adapter was approximately $70, Dell'Oro Group noted.
"The best piece of advice we can give people right now is: Don't just assume your existing 1 Gig solution provider is going to be your 10 Gig solution provider," said Robert Passmore, a research vice president at Stamford, Conn.-based Gartner Inc. "Look at alternative vendor resources if for no other reason than to get a fair price, because right now we're seeing substantial variation in pricing in the market. You need to keep your vendor honest."
Passmore also cautioned that a move to 10 GbE can be a major undertaking, involving not only the replacement of cables, network interface cards (NICs), servers and switches, but also a possible enterprise network redesign, especially if virtual servers are driving the need for the extra bandwidth.
"You're not going to lay this out the way you designed it five years ago," he said.
Here's what you need to move to 10 Gigabit Ethernet:
End users essentially have three cable/connector options with 10 GbE: Category 6 or 6a copper with RJ-45 connectors, twinaxial (twinax) direct-attach copper with SFP+ transceivers or multi-mode optical with an LC connector.
"Copper is cheaper, but you can't run it as far," said Dennis Martin, president of Demartek LLC, an analyst firm that operates its own test lab. "Typically, you can run copper within a rack."
Stuart Miniman, principal research contributor at The Wikibon Project, said users with existing data centers might have plenty of RJ-45 copper cabling that they can use with 10 GbE. Cat6 cabling can go 55 meters; Cat6a is good for 100 meters. Low-power twinaxial copper could work over 5 meters to 10 meters in a rack. But those building new data centers might want to "just jump in and go all optical," since the current road map for 40 Gigabit Ethernet and 100 Gigabit Ethernet doesn't support copper cabling, Miniman warned.
"While you might be saying, 'Well, I'm just going to put in 10 Gig today,' chances are whenever you put that infrastructure in place, you'd like it to last for the next five to 10 years, and it's a big question mark if you go copper that it's going to be supported outside of the rack," Miniman said. "For outside of the rack, you're probably just going to go optical."
Market-leading manufacturers such as Broadcom Corp. and Intel Corp. may rule the roost for a Gigabit Ethernet NIC and 10 Gigabit Ethernet NICs , but they're facing a new form of competition from the makers of storage network adapters.
The converged network adapters (CNAs) that are used with Fibre Channel over Ethernet (FCoE) also work in standard networking environments, as well as block-based iSCSI storage and file-based NAS storage. Brocade Communications Systems Inc., Emulex Inc. and QLogic Corp. are prominent manufacturers.
"It's very difficult for the Broadcoms and the Intels to move into the storage world because of all the certifications that have to take place, so what we're seeing Emulex and QLogic do is move into that Ethernet space," said Bob Laliberte, a senior analyst at Milford, Mass.-based Enterprise Strategy Group. "As things are converging, they're the ones getting the design wins from the server vendors for all the converged networking."
Emulex's manufacturer suggested retail price (MSRP) guide lists several dual-port options for its 10 GbE OneConnect Universal CNA (UCNA): NIC with direct-attach copper interface at $780 and NIC with short-reach optical interface at $1,690; iSCSI with direct-attach copper at $1,330 and iSCSI with short-reach optical at $1,920; FCoE with direct-attach copper at $1,775 and FCoE with short-reach optical at $2,695.
"It'll do Ethernet, iSCSI, FCoE. That's the whole idea behind all of this. In the future, when you're building out your data center, you'll wire once and then you can decide what type of service you want to run over it," Laliberte said.
Cisco Systems Inc., the market leader in Ethernet switches, now advises customers to purchase CNAs rather than NICs, even if they have no immediate plans to deploy FCoE.
"That just provides investment protection, and with the kinds of discounts offered, the pricing is very competitive" with 10 Gigabit Ethernet NICs, said Kash Sheikh, a senior manager for Cisco's data center solutions marketing team. He noted that users will gain the flexibility to converge their IP-based storage and Ethernet LAN traffic and/or their FCoE and Ethernet LAN traffic.
CNAs can also function as a TCP offload engine (TOE) to improve the performance of iSCSI storage and NAS storage.
Beyond NICs and CNAs, other 10 GbE connector options emerging include LAN on motherboard (LOM), CNA on motherboard and CNAs embedded as a mezzanine card in blade servers. Hewlett-Packard (HP) Co., for instance, integrated a 10 GbE NIC on server motherboards beginning in November 2008 and more recently, in June, announced a partnership with Emulex to integrate CNAs on server motherboards.
While Cisco Ethernet switches are the market leader, there's no dearth of competitors, including Blade Network Technologies Inc., Brocade, Extreme Networks Inc., HP and Juniper Networks Inc.
Although Cisco maintains its Catalyst line of Ethernet switches, its newer Nexus platform handles 10 Gigabit Ethernet and FCoE traffic. In the same way he recommended CNAs, Sheikh advised users to deploy "a switch that has lossless Ethernet capability, so there is investment protection," in the event they decide to move to FCoE.
"You have to plan ahead when you buy switches, and this is true whether it's Fibre Channel or Ethernet. It doesn't really matter what speed we're talking about," said Demartek's Martin. "The trick with buying switches is you need to buy enough to cover everything you need now plus some growth that you can foresee for the next few months or a year or two, so you don't have to go through the hassle of having lots of [inter-switch] links."
HP emphasizes it ability to smooth the transition to 10 GbE. The company's 6120 Series Blade switches, for instance, offer Gigabit Ethernet/10 GbE dual-speed connectivity and Virtual Connect Flex-10 to let customers divide 10 Gb capacity and use "connection-adjustable bandwidth."
The Brocade TurboIron 24X also has ports that can sense Gigabit Ethernet or 10 GbE to help customers in a state of transition. "This is a way potentially of directly connecting in the 1 Gig and the 10 Gig products on the same switch and giving them this connected, single SAN upgrade path," said Harry Petty, director of marketing for data center products at Brocade. "You could take that same product and implement it in an all-10 Gig scenario. But I think people rarely want to do a forklift upgrade."
The major storage vendors all offer support for 10 GbE with block-based iSCSI storage or file-based NAS storage, although it generally comes at a price premium. The Wikibon Project's Miniman said the tipping point for customers tends to happen when the cost is within 2.5 times of their existing infrastructure for a 10-time increase in bandwidth.