To translate AI capabilities into operational impact, utilities need to evaluate where AI workloads run. As workloads grow more complex and cybersecurity needs change, utilities need a hybrid approach that pairs the unique benefits of both cloud computing and dedicated on-premises and edge processing, bringing AI closer to where speed, reliability, and data sovereignty matter most.
On-premises and edge GPUs have become essential in this evolution, enabling utilities to accelerate grid modernization, automate field operations, and run complex simulations faster.
How do utilities move from AI ambition to operational reality? Industry experts addressed this and more in AI in Utilities: Real Use Cases, Real Infrastructure, Real Scale, a recent webinar sponsored by Dell Technologies and NVIDIA in collaboration with Utility Dive’s Studio.
Why Utility AI Pilots Struggle To Reach Production
After decades of roughly 1% annual growth, utilities are now facing skyrocketing demand — much of it driven by the AI boom itself. “They cannot build fast enough; they cannot hire fast enough,” said Ahsan Yousufzai, Managing Director of Energy at NVIDIA. Leadership has embraced AI to extract more value from existing infrastructure, but the parts of the business that make money — generation, substations, transmission, distribution — are fundamentally different use cases. That's where AI adoption gets challenging.
Many utility AI efforts have proven successful in localized pilots across one or two facilities, but scaling that success into grid-wide operational impact remains difficult. Moving from early validation to full production requires dedicated compute infrastructure capable of supporting diverse use cases, from weather forecasting to grid planning, while preserving data privacy and security.
Additionally, utilities are swimming in data, but turning it into insight is often hampered by in-house expertise that can’t keep up and regulatory and security limits that keep sensitive data out of the cloud. “The biggest challenges always come down to technology, people and data,” said Russell Boyer, Global Industries Utilities Chief Technology Officer at Dell Technologies.
The Real AI Use Cases Gaining Traction with Utilities
To understand utilities’ AI needs, EPRI ran a global survey through its Open Power AI Consortium, which includes roughly 350 organizations worldwide. Drawing on nearly 18,000 responses, they produced and prioritized a library of 261 use cases. The survey showed which operational use cases utilities are prioritizing.
"They're looking at use cases like outage detection, asset image analysis, wildfire and vegetation management, transformer overload analysis, and even moving out to edge-based cyber anomaly detection,” said Jeremy Renshaw, Director, Open Power AI & Quantum, at EPRI.
Utilities are weighing return on investment: how much these AI initiatives cost, and how much they ultimately save ratepayers.
Why Owning Your Compute Changes the Economics of AI
AI workflows once relied on humans to direct tasks like predictive maintenance. Now, reasoning models and autonomous agents coordinate entire workflows, consuming massive volumes of tokens that can drive up public cloud costs .
Delivering real value while keeping budgets predictable requires aligning workload placement with your consumption profile. While cloud offers flexibility, high-volume workloads processing billions of tokens often make far more financial and operational sense on dedicated on-premises infrastructure, especially when latency, reliability, or strict data regulations matter.
“Sometimes the ROI is easily justifiable when you compare [on-prem investments] to your cloud costs and cloud expenditures.” Matt Corbett, Director, Information Technology Enterprises Infrastructure & Networking, EPRI
The Right Workload at the Right Location
Compute placement comes down to three questions:
- Where is the data created?
- Where is it accessed?
- Where can the AI model take real-time action?
Starting in the cloud is common, but as utilities scale, they often need to move production models to where the data lives, including the edge. As field workers increasingly interact with digital versions of physical assets, on-site simulations that analyze and troubleshoot problems speed up power restoration and improve reliability.
That capability is critical in remote locations without cell signal or a hardline connection to the cloud. Running AI models directly on hardwired assets, such as a substation or transformer feeder, means operators can get real-time insights without relying on connectivity. It also strengthens security by keeping data and model responses on-site.
Ultimately, the answer isn't cloud versus on-premises. It's a hybrid mix shaped by project needs, where the data lives, and the constraints unique to each use case.
Tips for Launching Your First High-Value AI Deployment
- Start with a clearly defined problem and thoroughly map workflows before applying AI or automation
- Prioritize collecting the right data
- it’s foundational to everything that follows.
- Plan for the full project lifecycle, from fast, affordable ideation through production.
- Focus on business outcomes, not just technical capability.
- Right-size infrastructure with expert partners who know your use case.
- Once you've proven value on a scoped problem, scale deliberately.
- Tie ROI back to customer outcomes
- delivering more stable, reliable, and resilient power.
The Role of the Dell AI Factory with NVIDIA
Dell built the Dell AI Factory with NVIDIA so utilities can spend less time integrating and procuring components and more time improving operations. The validated framework lowers the barrier to entry for utilities without deep in-house developer benches. Partner ecosystems like the Open Power AI Consortium connect with domain experts for global engagement and value.
“It's not just about the hardware; it's about validating that full environment for the compute, the networking, the software, the models, the services,” Boyer said.
The architecture follows a "crawl, walk, run" approach: develop a model at small scale, validate the results, then move it to enterprise-grade infrastructure using the same software stack. NVIDIA's CUDA platform, standardized and compatible across hardware versions, runs consistently from desktop GPUs to data centers to the far edge so engineers can train and run models with the same tools regardless of deployment location.
Getting Placement Right Is the Path to Scale
The utilities that get workload placement right will move AI beyond the pilot stage. That means building the mix of cloud, on-prem, and edge compute that fits each use case, backed by validated architecture that scales from a single desktop GPU to a full production environment. At that point, AI stops being a series of disconnected experiments and starts becoming part of how the grid actually runs.
