In the opening keynote remarks at Gartner's IT Symposium in Orlando, Fla., Daryl Plummer, a distinguished vice president analyst at Gartner, said IT executives are spending 18 times as much on AI as they did just two years ago.
The problem is that companies are still hit with surprise bills. Because AI is a new frontier for many companies, some items are missing from their budgets. Small to midsize companies do not have the budgets of some larger tech firms, so every dollar counts.
TechTarget asked IT leaders about what they noticed with hidden costs in AI adoption. This is what they had to say.
What's the hidden cost of AI adoption that most executives don't budget for?
"The hidden cost makes me think of just the cost of experimentation. People will experiment and play, and that's how they'll start. They're not going to start with a perfectly defined project, so I think the hidden cost is just that—play exploration times.
The second is that people jump to tools to get the most value from AI in the long term; however, you really have to reimagine your workflows, your operating model and so on. So, to me, that's a hidden cost of getting value out of it because you really have to have a clear vision and picture of where you want to go, and that's hard. So, it takes iteration and time.
The third one is workforce preparation, which is more personal. For example, what's changing in your role? Are you equipped with the skills but also with the mindset around why some of these things are changing to be part of it? So maybe that's not as much about hidden costs of AI adoption, but I think adoption is getting to the point of value."
--Amy Farrow, CIO at Celigo
"[I think] it might be the context piece. We're in industrial manufacturing, so we hit this all the time: Our data, the way we say it, and HighByte PR are part of it, but our data is kind of a mess. We've never had a data strategy. So, what's happening in industry right now is that executives are asking, 'What are we doing for AI?' It makes it down to someone technical who understands factories and the data mess we have, and they say, 'We need to fix this before we can make use of AI.'
And when that doesn't happen—when you don't have that smart person who understands the data architecture of the factory is a mess—they just try to use AI on the mess of data they have, and you get very little because industrial data has very little context. So, you end up spending a lot of tokens and getting results that aren't super reproducible. You get a lot of hallucinations as AI tries to figure out the 20 years of data and decisions that were made on the factory floor that don't necessarily make sense even to a human. AI is just going to come up with an answer. And that's where you need the human in the loop. So, I think that's probably the biggest thing: AI companies don't do a great job of showing you the spend or token count for any specific task.
I think the one thing executives think is that 'We have data. Just set AI off on it, and we're good.' And it's like, no, you need some level of context and organization to make that reproducible and keep spending down."
--Aron Semle, CTO at HighByte
"I think tokens. So, even just usage and sprawl within an organization, people usually sign up for a subscription. They start using it as soon as it starts going to other groups, or other groups aren't paying attention to their actual token spend. There, it's almost like the model isn't designed properly to roll out, and then everyone starts attaching to it.
On the agentic front, people are setting up agents, and they're not doing it the right way. They're not looking at them systematically, so it's very inefficient and burns through tokens faster. I think some better ways or gauges are to separate it by group, so different teams are allocated a certain amount of funds, and then you can throttle or restrict certain teams from spending everything for the entire company."
--Chris Carreiro, CTO at Park Place Technologies
"The most recent one we all experienced is a token consumption cost. None of the CFOs had this as a line item in their Opex like a year ago, right? And that is just coming to the surface as the consumption, the model and the pricing of using AI have been changing. We're all kind of trying to figure out how to get the arms around it, what it really means and where to draw the line. And that's probably one hidden cost we didn't know, and we are all learning about it at the same time."
--Rajesh Arora, chief data and analytics officer at Principal Financial Group
"I think it comes back to context. So, for employees, we assume that everybody knows what's happening and that everybody could answer: What are our products? Who do we sell to? Who are our competitors? But so many people have different layers of information and context that really aren't documented, and that is incredibly important to how they make decisions.
When we add AI to the mix day-to-day, not having that documented in all sorts of ways, large and small, is incredibly costly for a business. Because if we let single agents roll through without real context, they just will burn tokens all the way through as they kind of sip information from various data sources, instead of really having a sort of navigable path where you've got all that information."
--Janelle Milodragovich, vice president of AI market strategy at Smartsheet
"I think all the AI love that our colleagues are generating out there and then giving it to cybersecurity and identity leaders to review. People come to me because I run competitive intelligence, and they say, 'Look at the app I built, and it tells everything about our competitors.' Now, you gave me three days' worth of extra work to review something you created in two minutes.
So, I think that's one of the areas that we have to watch out for because it's very cheap to create content, to generate things, but it's very expensive, and that's the hidden cost of just reviewing and cleaning up."
--Henrique Teixeira, senior vice president of strategy at Saviynt
Amanda Hetler is senior editor and writer within the Reader Engagement and Outreach team of Informa TechTarget.
