AI models are getting cheaper. For CIOs trying to control spending, that sounds like good news — and it is. But the model with the lowest advertised price isn't necessarily the least expensive to use. And choosing a model is only the beginning of what an enterprise will spend putting AI to work.
Recent reporting across TechTarget's AI and Data newsrooms illustrates why. More choices can help companies save money, but performance, infrastructure, data and governance all complicate the calculation.
Consider Red Hat, which is piloting a self-hosted open-weight AI model from China's Z.ai with as many as 1,400 engineers. About 70% of their inference traffic goes to open-weight models rather than more expensive proprietary frontier models.
Two ways to describe AI models
Open-weight models: Make their trained parameters available for organizations to download and run themselves or through another provider.
Frontier models: Offer some of the most advanced AI capabilities, often at higher costs
The distinction: These aren't mutually exclusive categories. A model can be both open-weight and frontier. What matters is choosing the right model for the work.
Red Hat isn't telling engineers they can't use expensive models. It's giving them alternatives and asking them to think about which model makes sense for the work. Engineers have a $300 monthly spending allowance for externally hosted frontier models, with exceptions when additional spending is justified.
Stephen Watt, distinguished engineer and vice president in Red Hat's office of the CTO, described the problem with relying too heavily on premium models. Although they're useful early on, "after a while, you'd be crazy to stay on it because it just eats all your margin," he told TechTarget's Patrick Thibodeau.
Red Hat's approach makes sense. Why pay for a more expensive model when a cheaper one can handle the job? But when the work calls for more advanced capabilities, paying extra might be worth it. The hard part is knowing which is which.
Depending on the job, a seemingly expensive model could be the more economical choice.
Cheaper isn't always less expensive
At cybersecurity company Xbow, tests found that some lower-priced models needed substantially more tokens to solve problems than proprietary alternatives. The extra processing reduced much of the expected savings. Depending on the job, a seemingly expensive model could be the more economical choice.
DataCamp ran into a different problem. The online learning company evaluated Google's open-weight Gemma model for its AI Tutor, but providers hosting open-weight models couldn't consistently deliver the response times and capacity the application required. Rather than move its tutor to an open-weight model, DataCamp is switching from Anthropic to OpenAI, which it considers a less expensive option for its needs.
In DataCamp's case, the model itself wasn't necessarily the issue. The infrastructure available to run it couldn't reliably support the application at the speed and scale required.
These examples make a straightforward price comparison much less straightforward. The price per token matters, but so do the tokens consumed, the results produced, the response time and the work needed to finish a task successfully.
What it takes to put AI to work
Even after an enterprise chooses the right model, it has to make that model work with its data and existing systems. CIOs have spent years trying to improve data platforms, governance and control over information scattered throughout the organization. AI makes that work more important, not less.
Metadata is taking on a more active role in that effort. Once used primarily to document data and explain what it means, metadata increasingly provides the context and controls AI systems need to use information properly.
For example, at Vanguard metadata supports AI tool data discovery, agent permissions and traceability. It helps systems not only find the information they need but also understand what they're allowed to do with it.
A model can be remarkably capable and relatively inexpensive, but if it doesn't have the right information, can't interpret it correctly or lacks appropriate controls, its capabilities won't count for much. Those requirements don't disappear because the model is cheaper.
“The governance of the data is a fundamental consideration that experimentation teams leave until the end, and then the use cases fail.”
Sergio Gago
chief technology officer, Cloudera
And governance isn't always something developers can add once an application is built.
Cloudera CTO Sergio Gago described the problem in an October 30 interview with TechTarget's Eric Avidon. A prototype can seem impressive until the team discovers that its data governance is inadequate for production. For instance, an agent might let one customer see another customer's information. If the flaw is built into the application, adding a safeguard later might not be enough.
"The governance of the data is a fundamental consideration that experimentation teams leave until the end, and then the use cases fail," Gago told Avidon.
Governance, security, data quality and access controls aren't new concerns for enterprise IT. But AI systems, especially agents that interpret instructions and take action, can require those controls to be more explicit and more closely tied to how the application works.
That work has a cost. Some expenses belong to individual applications; others are investments in shared data platforms and governance systems that support multiple AI deployments. They might not appear on a model's bill, but they belong in the broader calculation of what AI actually costs the enterprise.
That doesn't mean CIOs should shy away from cheaper models. Red Hat is showing how much sense it makes to give engineers choices, rather than paying for the most expensive model every time. The trick is knowing when spending more is worth it.
Choosing the right model is only part of the decision. CIOs also have to know what it will take to run it, connect it to the business and make sure it operates reliably and safely.
The best choice isn't necessarily the cheapest model, the most powerful one or even the one that performs best in a pilot. It's the one that can do the job properly, at a reasonable overall cost, without creating more problems than it solves.
James Alan Miller is a veteran technology editor and writer and lead editor for CIO news at TechTarget. He directs coverage of enterprise technology strategy, AI, software, data, infrastructure and the decisions shaping how CIOs manage increasingly complex IT environments.
