AI costs mirror early cloud computing challenges. Because usage scales unpredictably rather than under fixed contracts, traditional budgeting methods are ineffective and lead to surprise invoices that exceed planned budgets.
Hidden expenses emerge from multiple sources, including decentralized employee subscriptions, agentic AI performing complex multi-step tasks, API consumption and shadow AI purchases that bypass IT approval processes.
CIOs share their insight on how to control costs, including establishing AI cost centers with chargeback models, setting real time usage thresholds, monitoring autonomous agent activity, standardizing model selection and applying FinOps principles with predefined success criteria.
AI usage often starts small in many organizations, but that can change quickly, leading to surprising AI bills.
A department picks up a chatbot subscription, and a developer wires an API key into a prototype. The initial costs are low and don't show up as big numbers in any budget line item. Then it scales, and the free tools become enterprise licenses, developers push APIs to production, and agents start executing tasks on their own. To make matters worse, multiple business units end up buying overlapping tools without knowing the others exist.
For CIOs and finance teams that pay the bills, the result can be shocking -- AI invoices that exceed budgets. Cloud computing went through the same shift a decade ago. AI budget management is now following that path toward AI cost governance.
The rise of AI bill shock
AI bill shock follows a pattern that is familiar to those who lived through the early years of cloud computing, when the true cost of a service surfaced only after the invoice arrived.
GenAI costs behave the same way as cloud costs, scaling with usage rather than with a fixed contract, which is part of what makes enterprise AI budgeting so hard to pin down in advance.
The shift happened almost overnight at one university.
"In 2024, AI spending moved from a near-zero baseline to a recurring cost category very quickly due to decentralized purchasing," said Zach Lewis, CIO at the University of Health Sciences and Pharmacy in St. Louis. "Employees and departments were buying individual licenses for AI chat tools and productivity platforms before IT, security, procurement, and finance had a consistent process around approval, data protection, or budget ownership."
A similar pattern plays out well beyond higher education into enterprises across multiple industry verticals.
"Most CIOs I have spoken to in large enterprises have had difficulty budgeting for token usage and costs because there was no clear pricing and usage methodology when budgets were constructed late last year or early this year," said Joe Locandro, CIO at Rimini Street. "Most of the costs have been attributed to employees experimenting and learning how to do prompts for various tasks."
The increases land in predictable spots -- LLM API usage, GPU consumption, AI SaaS subscriptions, agent execution, and shadow AI spending finance never approved.
Most CIOs I have spoken to in large enterprises have had difficulty budgeting for token usage and costs because there was no clear pricing and usage methodology when budgets were constructed late last year or early this year.
Joe Locandro, CIO at Rimini Street
Where AI costs hide
Unexpected and increased AI costs can be hidden in several locations, remaining undetected until the accounting department receives an invoice.
Employee usage
Individual purchases are the easiest to miss.
"Employee tool subscriptions have been the hardest to track and predict," Lewis said. "It's harder to track when departments or individual users buy AI tools directly with departmental funds, procurement cards, or personal accounts."
Developers can also incur expenses for API calls, model testing and fine-tuning work.
Agentic AI
Agentic AI costs are particularly troublesome as they are driven by complex, multi-step work rather than simple licensing.
"Cost overruns will most likely come from agents performing complex tasks, including multiple lookups and API calls across various systems. These are more resource-intensive and will likely generate unexpected costs," said Locandro. "This, in turn, drives more costs as multiple runs are required to get it right."
Nicholas Merrizi, a Deloitte principal and AI infrastructure lead, points to a related blind spot. "Most organizations built their AI cost tracking around volume and per-seat licensing. The real blind spot, though, sits upstream, in large context windows, undefined retry logic, multi-agent loops, and inefficient caching," Merrizi said. "None of these show up as a distinct line item, but together they can quietly make a single task far more expensive than expected."
Data costs
The data piece of AI, which includes vector databases, storage and retrieval, rounds out the picture. Cost tends to scale with volume, though the exact relationship is unknown until used.
The new FinOps challenge
The FinOps process, which applies financial discipline to IT ops, was initially adopted for cloud consumption. It is now finding its way into AI in various ways, as CIOs struggle to determine the best approaches to apply FinOps to AI.
Traditional IT budgeting assumes costs are fixed and predictable. AI governance must account for spending that is consumption-driven, usage that fluctuates sharply, business users who can directly drive costs and agents that can generate costs without human involvement.
"Traditional FinOps fails because it manages static infrastructure on a lagging monthly billing cycle, while AI costs are driven by real time, unpredictable token consumption," said Erin Sawyer, chief financial and information officer at Certinia. "To protect margins and maintain agility, we must shift our focus from infrastructure-level tagging to real time application monitoring and immediate model usage limits."
Michael Corrigan, CIO at World Insurance Associates, has approached the challenge of AI ROI by extending the capabilities his company already has. Corrigan noted that his organization had developed strong FinOps disciplines around cloud hyperscalers, tracking consumption, forecasting demand, allocating costs to business units and optimizing usage patterns.
"We've been able to apply many of those same principles to AI token consumption," Corrigan said. "We now evaluate AI initiatives using the same discipline we would apply to any enterprise technology investment -- security reviews, business-case validation, expected ROI, adoption metrics and outcome measurement."
Not every cost shows up in a subscription or an invoice. "Organizations plan for technology and not for transformation," said Christine Park, chief AI transformation officer at Branch. "Access is not adoption. People need time to learn. The time it takes for people to build fluency and actually change how work gets done is part of the investment."
Five ways CIOs can prevent surprise AI bills
No CIO wants a surprise bill for AI. The way to prevent that unnecessary risk comes down to a handful of AI cost control practices.
1. Create an AI cost center
Nina Tatsiy, CIO at Quadient, shifted AI costs to a chargeback model once the company's adoption had matured beyond the pilot stage.
"We allocate AI costs to business units based on license ownership and utilization across platforms such as Copilot Professional and Claude," she said.
She added that, for finance, the approach provides much greater transparency than a centralized IT budget does.
"AI investment becomes directly linked to the departments responsible for delivering productivity improvements, making it easier to track adoption, forecast costs, optimize licenses, and measure ROI," Tatsiy said.
Billing usage data sent back to departments also revealed something Tatsiy didn't expect.
"The biggest insight was that different functions naturally used different AI platforms depending on their needs, which reinforced the value of our multi-LLM strategy," she said.
2. Establish usage thresholds
Waiting to see the bill at the end of the month is too late.
"We need real time visibility into usage patterns to understand the unit economics and, more importantly, to intercept runaway costs," Sawyer said. "Setting hard, real time usage thresholds acts as a control to limit any major financial liability."
Usage thresholds are also critical to ensuring business alignment. Locandro noted usage thresholds prevent runaway token use that does not align with value or productivity gains.
"If an employee wants to exceed caps or have them raised, management can monitor the expected value of the increase prior to token usage," she said.
3. Monitor agent activity
Agents are acting with greater autonomy and reasoning power on their own, which compounds the challenge of predicting costs.
"Unpredictability calls for a more nuanced approach to observability and central control," Merrizi said. "You cannot control or optimize what you cannot monitor, and monitoring AI spend takes a different approach."
4. Standardize approved models
Not every task needs the most expensive model. For many organizations, model standardization is one of the more direct forms of AI cost optimization.
"Standardizing on a fit-for-purpose model matrix ensures that simple tasks are automatically routed to smaller, cheaper models, vs allowing individuals to pick the latest, most expensive model," Sawyer said.
5. Apply AI FinOps principles
Corrigan traces World Insurance's clean track record to the deliberate rollout of AI tools.
"We haven't experienced any surprise spending spikes or unplanned token burn because we took a deliberate approach on which tools we deploy, by department or function, tied to specific use cases with predefined success criteria, and built in consumption limits," he said.
None of these practices eliminates AI spending. What they change is when a CIO finds out about it.
"The lesson I'd share isn't really about controlling costs; it's about measuring value from the beginning. For CIOs or organizations just starting their AI journey, my advice would be to focus less on the cost of AI and more on the cost of not using AI," Corrigan said.
Sean Michael Kerner is an IT consultant, technology enthusiast and tinkerer. He has pulled Token Ring, configured NetWare and been known to compile his own Linux kernel. He consults with industry and media organizations on technology issues.
