Enterprises make strides with private AI on-premises

Specialized models + governance = enterprise agent success

The latest generation of GPUs can still be pricey and hard to come by, as can memory amid a global supply shortage, but older generations of GPUs and even CPUs can now accommodate smaller, specialized and open models that have become more powerful over the past two years.

"When we started this, everyone thought we were going to need the best models for this, and in just the time it took us to plan to execute on -- and you've been able to see this across all AI -- all these models are getting so much better every day," said Adam Little, director of IT operations at New Belgium Brewing in Fort Collins, Colo., during a breakout session presentation.  

One example is a group of open models built for Nvidia's Omniverse platform, which New Belgium has used to visualize and optimize centrifuges in its brewery manufacturing pipelines. As the company moves to agentic automation for multiple parts of its brewing process and business, many models have now come into play, Little said, including models trained on New Belgium's internal data.

"A frontier model is not fit for purpose for everything," he said. "The ability to swap in and out and design different models for different use cases was important. The model I'm going to use to crunch controls data is very different from the model I'm going to use for preventive maintenance."

Little's co-presenter also said the use of multiple, specialized models had been important as his cybersecurity team built its first AI agent. Keeping the data used to run those models on-premises was equally important, as was the ability to use built-in VCF features to manage the internal agent workload.

"Being in a familiar [infrastructure], something we were going to manage on-premises, was important to us [for] security and control," said Mark Fournier, CIO at US Senate Federal Credit Union in Alexandria, Va., during the session. "[The IT ops team] has all the components from NSX and vDefend well dialed in, and they can say it's better than anything they feel they could execute on a public cloud. You don't really get any of that visibility or control."

Internally managed Deep Learning VMs (DLVMs) -- VM images packaged by VMware with VMware Private AI Foundations in VCF 9 -- also gave the cybersecurity team immediate visibility into the infrastructure while developing and testing its agent, which helped them "fail as quickly as possible, and then learn from it, iterate and fail again until we succeeded," Fournier said.

"Having the DLVM as a basic component allowed us to do that much faster than if we'd been doing a POC outside our environment," he said.

Broadcom declares private cloud 'tipping point'

Much has yet to be decided about where the bulk of enterprise AI workloads will run, but Broadcom officials were already declaring victory during presentations here this week.

Token cost is real, and it's starting to drive a significant change in the market.

"Token cost is real, and it's starting to drive a significant change in the market," said Broadcom VCF Division CPO Paul Turner during a plenary session presentation.

Turner cited a user survey conducted by Radius Tech in February and March, "in partnership with Broadcom," according to the fine print on a presentation slide. The title on the slide read, "The AI Tipping Point." "Among 1,800 professionals that were surveyed about the future directions in the market, 56% percent of them plan to run their production AI workloads on the private cloud, the survey found."Year over year, the preference for where they will run their AI applications dramatically changed," Turner continued. "[That represented a] 15% reduction in the public clouds used for AI operations, to only 41%. So people will still deploy some into the public cloud, but on a way smaller basis."

Enterprise AI outlook still at least partly cloudy

Elsewhere, however, analyst research directly contradicts this prediction. A December 2025 survey by IDC of 304 application platform decision makers found that organizations were deploying AI applications equally in both private and public clouds, at 49.9% and 50.1% respectively. Most respondents in that survey, 63.6%, expected that ratio to increase in favor of public cloud, while 21.6% expected it to stay the same.

The future is hybrid -- the pie is big enough for everyone to get a slice.

"I don’t think [there's] a massive push for repatriation, moving workloads from the public cloud [back] to the private cloud," said Matthew Flug, the IDC analyst who conducted the survey. "There are now just more workloads, and for some of those, the right environment won't be the public cloud. The future is hybrid -- the pie is big enough for everyone to get a slice."

This stance was also reflected anecdotally among VMware Explore speakers and attendees this week. The headliner speaker for VCF public cloud, Standard Chartered Bank, presented on a massive two-year project to roll out an updated, global private cloud spanning more than 200 clusters during the plenary session and a breakout session. But the bank's cloud infrastructure head said the company had not made any move to repatriate workloads from public cloud as part of that project.

"We are busy reviewing how we operate our workloads," said Hemant Rijhumal, global head of private cloud infrastructure and end user services at Standard Chartered, during a Q&A portion of the breakout session. "Right now, we're looking at our failure domains between public cloud and on-prem, and we've realized some workloads may have to go to the cloud, some workloads may have to come back on-prem. We have not done any major repatriation."

New Belgium Brewing also retains a mix of private and public cloud assets, including sales data, ERP and CRM applications hosted in Microsoft Azure, and a deployment of Azure Arc on-premises. The majority of its workloads have always been on-premises, rather than moving into and then back from a public cloud, Little said.

"We are looking at Copilot Studio to [manage] some agents -- this is just picking the right place for the right tool," he told TechTarget in an interview after his breakout session. "We have sales data in Azure, so we will build sales and probably marketing agents there. … Maybe long term, there could be an MCP connector or something, and we could do some of that on-prem if it's cheaper or there's more model flexibility."

Ultimately, few industry players hold as strong a view on private cloud supremacy as Broadcom. But for one analyst, IT leaders planning an enterprise AI strategy should take note of these discussions, regardless of where the overall market might be headed. 

"Many companies are taking a serious look at private cloud for AI. … The fear of 'tokenmaxxing' has driven a lot of conversations this way," said Rob Strechay, lead analyst at VB Intelligence. "It's still not cheap, but it can be significantly cheaper in the long run. If I were a CIO, I would be hedging my bets and looking for a way to make my [private] infrastructure AI-ready."

Beth Pariseau, senior news writer for Informa TechTarget, is an award-winning veteran of IT journalism. Have a tip? Email her or connect on LinkedIn.