GenAI data center infrastructure reshapes business processes
Building and retrofitting data centers to support GenAI is not a matter of IF but WHEN, so the game plan must include power needs, grid connectivity, model training and permitting.
Businesses are rethinking their data center infrastructure from the ground up to support generative AI capabilities. Traditional data centers focused on optimizing storage arrays, VM placement on CPUs, ethernet connectivity and the provisioning of relatively low-power systems. AI data centers shift the focus to optimizing data pipelines; AI training, fine-tuning and inference across GPUs and high-bandwidth interconnects; and the provisioning of significantly higher-capacity power delivery systems.
"AI has completely transformed the role of the data center," said Jennifer Skylakos, managing partner and head of sustainable infrastructure and energy at consultancy DHR Global. "It's about delivering sustained, high-density compute at a scale that traditional facilities were never designed to support."
There has been a major shift in the engineering center of gravity, said Jim Piazza, chief AI officer at IT managed services provider Ensono. The traditional data center bottleneck was data storage throughput and virtualization density. Bottlenecks in AI data centers occur when resilient and flexible parallel compute capacity is taxed by new data pipelines running across low-latency, high-bandwidth GPU fabrics and by high-bandwidth memory connectivity among racks. Also, rack densities are increasing from 10s to 100s of kilowatts with the emergence of liquid cooling systems.
However, uncertainty about future AI ROI is tempering insatiable demand for GenAI. Still, businesses are realizing profitable and sustainable new business and technical models despite concerns over new data center capacities, financing, grid connectivity and pushback from affected neighborhoods.
Most GenAI data center expansion by hyperscalers will move forward thanks to backing by long-term business contracts and sovereign AI initiatives, although power and permitting constraints might delay deployment schedules, Piazza conjectured. For smaller enterprises, however, he expects AI infrastructure buildouts will moderate. "Many organizations are reassessing whether they need to own [AI model] training infrastructure versus leveraging cloud or colocation models," he said. "Over the next 12 to 24 months, we'll likely see some right-sizing as inference efficiency improves and model optimization reduces raw GPU demand per workload."
GenAI data center game plan
GenAI data center strategy requires fundamental changes in virtually every aspect of physical IT, in addition to developing new supporting business and technical processes. Business leaders need to consider several factors in their GenAI data center strategy, including the following:
Fiber and grid connectivity must be sufficient for existing data centers and new builds, while businesses battle for permit approvals from skeptical local communities.
Traditional, relatively static storage array architectures are being reshaped to support on-time delivery for training, inference and fine-tuning AI models with appropriate governance.
The evolving capabilities of AI chips require flexibility when deciding between established chip vendors and cost-effective yet less-established vendor alternatives.
Virtually every layer of the AI stack is rapidly accelerating functional depreciation or experiencing supply chain shocks that introduce financial uncertainty when deciding to upgrade AI models, chips, memory, storage, power and connectivity systems.
Businesses must plan for a multifaceted infrastructure that supports training, inference and fine-tuning AI models as teams experiment with use cases that deliver actual ROI or competitive advantage.
Business leaders also need to assess how new GenAI techniques and supporting processes can produce value while adhering to appropriate security, governance and safety guardrails.
Productivity and internal automation. Copilots, chatbots and agentic tools are getting better at summarization and analysis. But they are also introducing new problems related to hallucinations, deficiencies in large language models (LLMs) and contextualization. These issues require subject-matter experts in collaboration with business teams to refine AI quality processes.
Customer engagement. GenAI-powered support channels are automating an increasingly larger number of customer requests, but they're also introducing legal and brand risks arising from inaccuracies and negative customer experiences.
Embedded agents. Internal agentic tools are extending the capabilities of traditional automation across more types of unstructured data in domains, such as procurement, finance, compliance and customer management, but they're also creating new financial, legal and security risks.
Domain-specific inference.GenAI tools are getting better at mapping data from unstructured documents and disparate apps and services to support documentation in regulated industries. But expert oversight is often required to prevent regulation, compliance and safety issues.
GenAI will have a profound effect on data center processes.
GenAI data center transformation
Transforming a data center to accommodate GenAI starts with an assessment of existing data assets, including their purpose, formats, metadata, compliance requirements and biases. High-quality data produces better quality GenAI outcomes to help determine suitability for new workflows and supporting infrastructure requirements.
GenAI workflows can be categorized into data management, model optimization and model operation processes. Retrieval augmented generation (RAG) is the workhorse for data management and can inform both training and inference processes. Model optimization techniques include training a new model from scratch, distilling a large model into a smaller one and fine-tuning an existing model. Each AI model in operation has different infrastructure requirements.
RAG is the best choice for 80% of enterprise deployments, said John Bradshaw, field CTO for cloud in Europe, Middle East and Africa, at distributed cloud and security platform provider Akamai Technologies. "RAG is … cheaper, faster to deploy, doesn't require machine learning engineering talent, which is scarce and expensive, and keeps the data governance story simple," he explained. "Your proprietary data never touches a training pipeline."
Distillation and adapter-based techniques, such as LoRA and QLoRA, essentially train a smaller model optimized for performance or specificity in conversation with a larger model, Bradshaw noted, adding that they offer about 80% of fine-tuning benefits at about 20% of the cost and complexity. "This is where I'd bet the volume lands for enterprises that need more than RAG but can't justify full training pipelines," he said.
In contrast, fine-tuning a small language model makes sense for about 15% of use cases in which domain specificity or edge deployment justify it, Bradshaw surmised. Training an LLM from scratch only makes sense for less than 5% of use cases. Most businesses dramatically underestimate ongoing fine-tuning costs that incur additional expenses for evaluation, drift monitoring, retraining cycles and the talent to manage it all, Bradshaw added. "Foundation model capabilities are improving fast enough that today's fine-tuning advantage gets absorbed into next quarter's base model," he said. "The enterprises I'd expect to train are those where the model is the product or where regulatory and sovereignty constraints genuinely prevent using third-party APIs."
Large-scale GenAI model training requires facilities with access to abundant, ideally low-carbon energy, because sustainability and energy efficiency are critical constraints, said Masahisa Kawashima, IOWN technology director at NTT. Fine-tuning doesn't require the same compute capacity. It's more constrained by sovereignty, governance, security and low-latency integration with other business applications and data assets. As a production layer, inference requires planning for unpredictable, highly variable demand across multiple service endpoints -- often a good fit for elastic cloud resources that support dynamic scaling and pricing models.
GenAI infrastructure constraints
GenAI infrastructure expansion faces multiple physical, technology and supply chain constraints that are constantly evolving.
What used to be a facilities discussion is something CEOs and boards are actively talking about before approving expansion plans.
Jennifer SkylakosManaging partner, DHR Global
Physical
The biggest physical constraints for building new data center infrastructure are energy, permit approvals and fiber connectivity, with energy being the most pressing, according to Kawashima. But permitting risks are becoming a growing challenge, fueled by local community animosity toward rising power and water costs and environmental concerns.
A 2026 Heatmap Pro analysis found that at least 25 U.S. data center projects were canceled due to public opposition in 2025, up from six in 2024 and just two in 2023. "Energy availability has now moved into executive conversations," DHR Global's Skylakos said. "What used to be a facilities discussion is something CEOs and boards are actively talking about before approving expansion plans."
Technology
The cost-to-performance ratio of chips and interconnects is decreasing more rapidly than traditional IT assets, which runs the risk of stranding capital if enterprises overbuild today, Ensono's Piazza said. Meanwhile, innovations in AI model training and inference techniques are further reducing these costs.
A 2025 Stanford report found that AI inference costs dropped more than 280-fold between 2022 and 2024, while annual hardware costs declined 30% and energy efficiency improved 40%. Meanwhile, open weight models that eliminate licensing costs have also narrowed the gap compared with commercial models. The implication is that companies need to evaluate the early GenAI adoption advantage today versus the cost advantage of cheaper infrastructure and supporting tools in the future.
"The practical takeaway for enterprises is to avoid locking into long-lived, oversized builds based on today's peak assumptions," Piazza advised. "Modular deployment, phased expansion and hybrid consumption models reduce stranded capital risk and are a more resilient approach."
Supply chains
The insatiable demand for GenAI infrastructure has driven massive price spikes in data center memory and storage equipment that indirectly affect computers and mobile devices. Counterpoint Research's February 2026 Memory Price Tracker reported that memory prices soared 80% to 90% between Q4 2025 and Q1 2026. It will take years for suppliers smarting from earlier memory gluts to build up supporting capacity.
Meanwhile, acquisition timelines for power generation equipment, such as gas turbines and diesel generators, are stretching up to seven years. Renewable and battery alternatives face shortages from western manufacturers and a constantly shifting tariff and compliance landscape. While GPU, memory and power infrastructure costs have spiked recently, Piazza said, much of the increase reflects demand shock rather than a permanent baseline. These prices will likely moderate over time as supply catches up to demand and economies of scale take effect for producers.
Enormous thirst for water is among the physical GenAI data center constraints.
Planning a hybrid portfolio
To counter real estate constraints, a common approach is to retrofit existing data center facilities to support additional capacity, Piazza suggested. Upgrades could support higher rack power densities, more efficient cooling technology, network architectures and data processes before committing to new chips. "GenAI is shifting data centers from storage-optimized environments to power- and bandwidth-optimized environments," Piazza said.
Hybrid planning also includes capital allocation, cost assumptions, business priorities and governance alignment based on which processes need to run in-house versus on the emerging AI cloud. "This isn't just a technology decision; it's a capital allocation decision," Skylakos said. "Enterprises are reassessing what must be built in-house versus what can live with hyperscalers, and that requires alignment around risk, control and long-term cost."
Himanshu Jain, a partner in the digital and analytics practice at the consultancy Kearney, recommended a four-pronged approach to upgrading data centers to accommodate GenAI:
AI pods retrofitted into existing on-premises data centers for predictable and latency-sensitive use cases that benefit from the data's close proximity.
Dedicated larger capacity colocation or new build-to-suit facilities that offload complex processes to data center management partners.
AI cloud agreements for burst training at scale that reduce the risk of equipment obsolescence.
Edge inference infrastructure in facilities to support real-time apps, resilience and improved governance.
Planning for edge AI data centers
Edge AI infrastructure design is rapidly evolving, much like more centralized or cloud infrastructure but in slightly different ways and with different constraints. Smaller, yet performant AI models, combined with better hardware, mean more applications can run closer to where they're required. And similar to edge computing, practitioners have a variety of analogies and framings to shape the AI edge.
The ability to tightly and securely couple edge data centers with diverse cloud computing environments will be fundamental.
Masahisa KawashimaIOWN technology director, NTT
The core function of edge AI data centers "is not to act as large-scale computation centers," NTT's Kawashima said, "but rather to operate as true data centers in the literal sense or as 'data springs' -- places where valuable data is collected or generated and made available for AI applications."
To unlock the value of fresh data at its source, however, edge facilities must be seamlessly connected to cloud-based compute centers. High-speed, low-latency network connectivity enables on-demand, secure access to distributed computational resources. Data is anchored at the edge, while compute capabilities are dynamically accessed through secure, high-performance networks. "The ability to tightly and securely couple edge data centers with diverse cloud computing environments will be fundamental to building responsive, scalable and AI-driven services in the future," Kawashima said.
All-optical connections could dramatically shape the future of data center infrastructure. Today's data moves between electrical and optical signals at every layer of the infrastructure stack, creating latency and unpredictability that hinder deployment models and use cases. Kawashima, who chairs the Innovative Optical and Wireless Network initiative, said progress is being made on future data centers, wide-area networks, server racks and AI chips.
"Rather than centralizing everything in a few mega-facilities," he explained, "we create a networked constellation of sustainable data centers that together deliver massive computational power. This shift from scale supremacy to collective capability will be essential for building scalable, resilient and energy-efficient AI infrastructure for the long term."
There's also recent speculation about AI data centers deployed in space. Some visionaries have imagined thousands of orbital data centers, free from earth-bound permitting and energy constraints. But engineering challenges include launch costs, latency, heat dissipation, orbital decay and extremely large solar arrays. And just like on earth, it all comes down to energy, according to Philipp Jung, a senior partner in the digital and analytics practice at Kearney. "GenAI data centers," he said, "will require significant advances in space-based power-generating infrastructure before orbital deployment becomes feasible."
George Lawton is a journalist based in London. Over the last 30 years he has written more than 3,000 stories about computers, communications, knowledge management, business, health and other areas that interest him.
