AI agents can leave CIOs guessing what the work will cost until the bill arrives.
A single task assigned to an AI agent can trigger repeated model calls or bring other agents into the work, consuming more tokens and computing resources along the way. That variability makes it harder for CIOs to forecast spending and set effective limits.
Forrester principal analyst Kevin Ogunsua makes the challenge sound manageable: Set limits on agents, hold vendors accountable and connect spending to business results. His background spans software development and accounting, including 10 years implementing financial systems as a consultant—experience that informs his emphasis on budgets, cost allocation and vendor accountability.
In the following interview, Ogunsua explains what CIOs should demand from vendors, how to handle an agent that exhausts its budget and why managing AI spending requires decisions that reach beyond IT.
Why is it so difficult to predict what an AI agent will cost to run?
Kevin Ogunsua: Before agents, most of IT was very deterministic—you knew what you were going to get. With agents, it's different. Every time you ask it a question, you're not necessarily going to get the same answer. That's because they will go in, figure out what the request or prompt is, and then determine how many times they must call the model.
Also, agents can call other agents, so you never have a good way of knowing how many tokens you’ll have to pay for. You don't know how much compute, how many networks, how much storage it is going to consume because it could vary depending on the time of day, the request you get and the model you call.
How can CIOs bring unpredictable AI agent costs under control?
Ogunsua: First, identify the problem. It usually boils down to about five things: latent cost, unclear ROI, skills gaps, shadow AI and price changes from hyperscalers and model builders.
The other question is, what’s the root cause? This can include challenges in frameworks, enterprise misalignment, overlooked change management and undefined outcomes. Once you figure out the problem and its root cause, you can move on to solutions.
To come up with a solution, you must determine what value you want to get from AI. We use a value matrix that breaks value down into two dimensions. One is financial outcomes, which captures revenue creation, operational efficiency, cost reduction and risk optimization. The other is how you pull the levers: productivity, customer engagement or transformation.
To help control costs, we use what we call the Big T framework.
How does your Big T framework help CIOs manage AI agent spending?
Ogunsua: The framework explains how AI token consumption can dramatically increase depending on your request and the model calls. Then you can actually get an idea of the cost before the invoice arrives.
Here’s an example. You make a request in Gemini, Claude or even an open-weight model. When you prompt it, or an agent calls, that first part is the request—the input. Here, you must look at the prompt. If that prompt is too big, it would consume too many tokens, especially if it has to interpret it.
Second, you must consider how many calls the request makes. If it's making 20 model calls, that's 20 times the number.
The third part is each model could call an agent, and each agent could call other agents. Do you see how this can become a problem? You've got requests multiplied by the number of model calls per request multiplied by the number of agent calls. So, you're not too sure what you're going to get. It's going to spiral up.
The solution is to bound each thing. First, make the request small and try to cache. Also, you want to reduce the number of calls that the model is calling, and then the number of agents.
How can CIOs track an agent's model calls, particularly when it's part of a vendor platform?
Ogunsua: Ask the vendor to have what we call observability, which lets you go in and query the calls. Many vendors now include that in their packages. You need to know what the prompt is, the number of tokens—the input tokens and the output tokens—and the number of processes. Then, you can budget.
“Business people should tell their technology providers that they need to be able to observe, query, budget, bound and control the data coming out of these models.”
When you budget, you might say, "Good, we've analyzed this before it runs. We know there are two or three calls. When we make calls, we don't want to use more than 1,000 tokens or thereabouts." Then, you can set that limit in the harness—the environment that allows your agent to run.
Business people should tell their technology providers that they need to be able to observe, query, budget, bound and control the data coming out of these models.
What if a customer service agent reaches a spending limit halfway through resolving a case?
Ogunsua: Back in the day, when I used to develop code, we had something called the handler. When I was with another company where I implemented systems, we did a lot of design thinking, and we also had something called happy path and unhappy path. What you're talking about is an unhappy path. When you're designing a system, you cannot assume it will always go the right way.
At every point, you need a handler. You must consider problematic scenarios like the one you just explained. That will be part of a test case. Then, there are two things you can do. In your example, you can route it to a human in the loop—that's one way. The other thing is have a handler that does that and says, “Okay, let's see what this is,” and then figure out what the impact is if this isn’t solved right now. A lot of times, when you have an issue, you must bucket it into various areas by importance. Does it need to be fixed now, or is it non-important?
If it’s urgent, you probably want to increase the agent’s budget because it’s critical to the organization. But if it's something not very impactful, you may leave it unresolved for now. Of course, if it's something in production and customer-facing, that’s obviously urgent.
During your presentation at Forrester's Technology & Innovation Forum, an attendee described receiving a surprise OpenAI bill. How can CIOs avoid similar situations?
Ogunsua: I get those questions all the time from executives, and I tell them they need to change their procurement and contracts.
You want to look at the total cost of AI. You have to think broader because you're going to cut across businesses. That means having a conversation with your CFO to look at the general ledger and allocations: We have this cost that goes across the organization. How do we allocate it to different business units, departments or people?
Once you do that, you put a budget around it, look at actuals and look at the variance. Quarter by quarter or month by month, you tweak your allocations, your controls and your harnesses.
Then you invite the procurement person into the room. You might have to have a conversation with your CEO and the board to say we need to change the way we write our contracts. You make them outcome-based, flex-based or hybrid-based, and you tie the vendors to that.
If your provider says, “We can increase your revenue by 20% over two years,” you go, “Okay. Let's talk to the procurement folks. Let's look at the contract and put your money where your mouth is.” You have a baseline for your KPI and revenue, and you tie payments to revenue increases.
You also ensure there's training from a change management perspective. You may even have to change the compensation of procurement and other staff. A lot of the time, it's not the technology that fails. It's the people or the organization that prevents an initiative from succeeding.
What's the first thing a CIO should do to get their AI spending under control?
Ogunsua: First, understand what your business issue is. Look at your vision. Define a KPI for your use cases. Make the big bets. Then conduct a maturity and gap analysis.
Next, expand your IT spending model to include non-IT costs because, remember, AI is not just IT. Then you can launch what we call a strategic value office to determine what the value is. It's not really a project management office. We've found in our research that if you want to generate value, you need a strategic value office typically comprising IT finance, enterprise strategy, governance, enterprise architecture and strategic portfolio management.
But it's very important that you define your vision, your value and where you want to go. Have some good anchor bets, conduct a maturity and gap assessment, and then launch this initiative to help you optimize costs.
Tim Murphy is a reporter covering IT strategy for Informa TechTarget, with a focus on IT leadership, governance, workforce skills and AI strategy. His enterprise technology coverage has earned multiple Azbee Awards.
This transcript was edited for length and clarity.
