Maksim Samasiuk - Fotolia

Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Enterprises mirror hyperscale data centers to build new infrastructure

IT pros offer tips to help enterprises emulate hyperscale data centers like Amazon Web Services, Microsoft Azure and the Google Cloud Platform in their own organizations.

The rise of large cloud providers like Amazon Web Services, Microsoft Azure and the Google Cloud Platform has led to a new kind of infrastructure, and smaller enterprises are learning how to emulate these hyperscale data centers.

These massive cloud providers now run data centers at a scale never seen before, so they have had to take a different approach, said Dan Conde, an analyst at Enterprise Strategy Group, based in Milford, Mass.

The result has been the creation of hyperscale data centers that are based on low-cost white box switches and centralized software-defined networking (SDN) controllers.

Conde said 66% of organizations surveyed recently by ESG said they aspire to emulate the network infrastructure designs of the hyperscale cloud service providers in their own network architecture.

"Organizations looking for more agile ways to run workloads, segment networks more efficiently and save money on high-cost proprietary switches can learn a lot from the hyperscalers," Conde said.

Hyperscale data centers are deploying innovative technologies today -- among them open source, efficient power, microsegmentation, SDN and software-defined data center architectures -- that could be commonplace in the enterprise networks of tomorrow. And while the average enterprise won't run a data center operation the size or scale of Amazon Web Services (AWS) or Azure, they can borrow techniques from the hyperscalers to run their operation more efficiently.

New style of networking

[I]t is crucial to have a vendor that's engaging and willing to work with us to resolve expected and unexpected issues.
K.C. Wangassociate professor of electrical and computer engineering and networking CTO, Clemson University

For many different reasons, organizations are using this new style of networking to run their applications.

K.C. Wang, associate professor of electrical and computer engineering and networking CTO at Clemson University, said Clemson has been using both the Big Cloud Fabric and the Big Monitoring Fabric from Big Switch Networks for the past few years.

Today, Wang said all of the university's administrative and business applications, including its learning management system, run with Big Cloud Fabric coupled with VMware servers.

"While cost is a factor, it's not the only factor," Wang said. "Working with an early stage technology like software-defined networks, it is crucial to have a vendor that's engaging and willing to work with us to resolve expected and unexpected issues."

Wang said Clemson has also deployed the Big Switch solution in the university's disaster recovery facility in California, adding that the two facilities connect over the Internet2 network.

hyperscale data centers gain cloud market share

Network managers looking to manage hybrid cloud computing will find that as they move workloads out to the cloud, they can delegate some network management to cloud providers, but also keep control of the overall networking capability in-house. This may be by using a cloud version of the networking solutions they use on premises or by using VPNs that stitch together networks of different cloud providers.

Webscalers and the new right-sized data centers

Andrew Lerner, a research vice president at Gartner who covers networking, referred to enterprises that emulate hyperscale data center providers as "webscalers." These are the companies that don't need to scale the way AWS, Microsoft and Google do, but they can integrate other practices such as relentless automation and save money by using more commoditized switches and centralized network administration.

In fact, both Lerner and Gartner analyst Joe Skorupa estimated that by 2020, 40% of global enterprise CIOs will have initiated a corporate webscale initiative, which represents a significant increase from fewer than 5% today.

To succeed along the lines of a hyperscale data center, network managers must fundamentally change how their teams think about risk in relationship to the network, Lerner said. The mentality of "play it safe, don't risk an outage," was effective during the relatively static business models of the past two decades, he added, but that motto is no longer viable during the disruptive times of today's digital business transformation.

"By using white box switches, if hardware goes down, these low-cost switches can more easily be swapped out, and it won't cost the organization that much," Lerner said.

Network adminstration made simple

Joshua Bowers, systems engineer for Heathcare Realty Trust, a real estate investment trust (REIT) based in Nashville, Tenn., said the main reasons his company deployed white box switches and software from Cumulus Networks was to save money and simplify network administration.

Bowers said on a typical 10 Gigabit Ethernet switch, the company saves $8,000 to $10,000 on the cost of the switch, including savings on licensing fees over three to five years.

"The other thing to remember is that we're a small shop; I'm the only network engineer," Bowers said. "With Cumulus, I can manage all the switches from one single location."

It's also much easier to deploy new switches across the company's more than 60 branch locations. Bowers said he can now drop-ship the switch and hire a local tech support operation to come over and plug in the new switch. Once the tech plugs in the switch, Bowers identifies the switch on the network and configures the new switch remotely.

"Once I see the new switch come online, I can send over the configuration," he said. "And if there's a network failure, for the branches, we can send out a new switch the next day. For the main data center, we have plenty of extra white box switches right here. We always keep spares."

While there are many benefits, Bowers acknowledged that he went through a fairly steep learning curve in the beginning to integrate Cumulus with a Linux environment.

"For larger organizations that may have some Linux experts on hand, there shouldn't be any problem," Bowers said. "But I had to learn the Linux environment from scratch."

For example, Bowers said Cisco IOS makes configuring Layer 2 technologies like virtual LANs (VLANs) and trunk ports simple. In Linux, configuring Layer 2 technologies requires more technical expertise. While Bowers found working in Linux cumbersome at first, Cumulus has since simplified Layer 2 configurations by offering a "VLAN-aware" feature in its most recent versions.

On the other hand, Bowers said Cumulus integrates well with Linux automation tools like Ansible, which lets him automate networking services like Border Gateway Protocol and bring ESX hosts online fairly easily. Cumulus also has a tool called Network Command Line Utility, which Bowers said presents a more traditional, Cisco-like command-line interface and simplifies configuration of core network technologies even further to help speed adoption for network engineers with less Linux experience.

"We started down this path in January 2016," Bowers said. "At this point, we're running Cumulus in our data center and at 10 of the branch facilities, but plan to pick up the pace in the months ahead."

By borrowing techniques from the hyperscale data center approach, network managers save money and ease network administration, but they can also maintain some sense of control at a time when corporate IT feels it is losing control to the major cloud providers.

Although many organizations may aspire to be like a hyperscale provider, the vast majority does not have the scale or the skill sets to replicate those organizations. Companies need to carefully assess their situations and determine what works best for their enterprise. 

Next Steps

New hyperscale infrastructure

What can be learned from hyperscale developers

Beyond hyperscale cloud services

This was last published in August 2017

Dig Deeper on Data Center Networking