putilov_denis - stock.adobe.com
Why GenAI infrastructure optimization starts with the network
GenAI deployment requires optimized infrastructure. The networking aspects, such as workload placement and the type of deployment model, could determine GenAI performance.
An optimized IT infrastructure is a necessary foundation to support business operations and future innovation. This is true for legacy architectures, but it's also pertinent for infrastructures with emerging technologies that introduce new difficulties and errors.
Take generative AI, for example. The novelty of GenAI hinges on its ability to create original content and resources for businesses. But it can also provide misinformation, create management difficulties and accrue higher costs, among other pain points. Ideally, an optimized GenAI infrastructure addresses these concerns and supports businesses for years.
"The impacts [of GenAI] are quite profound," said Matt Eastwood, senior vice president of research at IDC, during a session at IDC Directions. "IT buyers are going through some pretty significant considerations around how to build an IT infrastructure that will sustain itself for the next decade or so."
In his presentation, Eastwood outlined how enterprises can optimize infrastructure for GenAI. Because GenAI infrastructure and data rely heavily on the network, organizations should consider networking factors, such as hardware costs and deployment models, and plan how to implement sustainability for energy efficiency.
GenAI adoption to boost network switch market
Enterprises should evaluate economic considerations during the beginning of an infrastructure optimization project, Eastwood said. Organizations plan to invest heavily in GenAI. In January 2024, IDC released a report, "Future Enterprise Resiliency & Spending Survey," which surveyed 881 IT professionals. An approximate 41% of respondents said they've already begun to invest significantly in GenAI.
Enterprises might allocate a large portion of that budget to GenAI-based switches. A February 2024 report from IDC predicted the AI data center switch market will increase from $41.9 million in 2023 to $1 billion by 2027. This trend appears to indicate a preference for on-premises GenAI deployments, but enterprises will scale GenAI in cloud environments as well. Eastwood said enterprises will deploy GenAI workloads between hybrid environments that include the network edge, public cloud and third-party cloud environments.
"This is really about one continuum of interconnected workloads that stretches from the core to the edge," Eastwood said.
Networking workload placement for GenAI infrastructure
As organizations plan to build GenAI infrastructure, they should decide how to place workloads in the architecture. Considerations include whether to build on or off premises and whether to use edge or core networks.
Workload placement considerations for GenAI infrastructure also include sustainability. As organizations modernize decades-old infrastructure to make way for GenAI, Eastwood said they've decided to build eco-friendly infrastructures that focus on sustainable lifecycle management, energy efficiency and carbon footprint reduction.
"In the end, the goal is an eco-friendly infrastructure that's energy-efficient, resource-optimized and potentially more modularized," he said.
Cloud networks can support sustainability, and many organizations plan to scale GenAI infrastructures in the cloud. An approximate 53% of respondents from IDC's survey said they planned to use private cloud on premises or public cloud off premises.
Edge IT has also emerged as an alternative deployment model. Around 20% of respondents said they planned to deploy GenAI environments in the edge. The choice between core networks or edge networks will depend on whether organizations want to harness the raw power of the core or prefer the instantaneous speeds of the edge, Eastwood said.
GenAI to contribute to multi-cloud networking growth
According to IDC, GenAI investments will drive multi-cloud networking use. Almost 90% of respondents said they were either actively implementing or had plans to implement multi-cloud networking in 2024.
The forecasted growth of the multi-cloud networking market correlates with most organizations' plans for GenAI infrastructure optimization. An approximate 66% of IT professionals said AI and machine learning workloads are one of their top use cases for multi-cloud networking.
"Fast data and efficient networks link apps and data," Eastwood said. "We're starting to see more awareness [that] AI's not just about compute -- it's [also] about the network."
Networking considerations for GenAI infrastructure optimization
An AI infrastructure encompasses a wide range of components, but the networking portion is one of the most crucial pieces. As enterprises build and optimize infrastructures for GenAI, they should evaluate various factors, such as cost considerations and sustainability imperatives.
The workload placement of a GenAI infrastructure is also important, as deployment location -- either public cloud or private cloud on premises -- could determine how enterprises build powerful, sustainable infrastructures. However, IDC's research indicated organizations will deploy and interconnect GenAI workloads in hybrid environments that include multi-cloud networks.
"IT's future is defined by [AI-powered] dynamic apps that are going to live at the core [and] edge," Eastwood said. "They're going to leverage massive amounts of data, be built using hybrid cloud as the operating model and they're going to do it sustainably."
Deanna Darah is associate site editor for TechTarget's Networking site. She began editing and writing at TechTarget after graduating from the University of Massachusetts Lowell in 2021.
