AI ROI: Focus on outcomes, not hours saved

In today's world, there is a divide -- humans or AI. What about if humans worked with AI, rather than being pitted against it?

Dave Coplin, chief envisioning officer and author of Business Reimagined and The Rise of the Humans, discussed how technology is enabling humans, not replacing them, in his keynote address at Info-Tech Research Group's LIVE event in June 2026 in Las Vegas.

Having spent more than 30 years in technology at companies such as Microsoft and Apple, Coplin now connects non-technical people with the potential of technology. He said that businesses do not have the right relationship with AI and should focus on effectiveness rather than efficiency when measuring AI value.

"I want you to think about measuring time spent. ... I don't care that you've saved time with automation ... Tell me, what you're going to do with the time. How are you going to deliver more value to your customers? What are you going to do to engage your employees?" Coplin said in his presentation.

He said that's the real measure of AI, and businesses need to use the right metrics for success.  

He also said shadow AI shows excitement in the business and that people are finally embracing the technology, proving that AI can help businesses harness employee energy when used appropriately.

TechTarget sat down with Coplin after the session to ask him some additional questions.

Editor's note: The following transcript was edited for length and clarity.

You mentioned that businesses are measuring AI ROI incorrectly, using metrics such as time saved. How do you update your measurements to be more meaningful and tangible to the board?

Dave Coplin: Every single digital transformation we've ever tried has failed for the same reason or struggled to articulate what it's delivered because we're measuring in hours saved or head count reduced. That's easy to measure. Instead, what we should be measuring is the outcome -- the quality of the outcomes -- not even the output. Not that I've made 15 more widgets since I created this thing, but my customers were delighted, or it helped my employees become engaged.

I think the single biggest challenge facing managers today is equipping them to measure outcomes. The problem with outcome versus time is that time is tangible, while outcomes are subjective. I've seen this play out in several organizations, where managers can handle that level of ambiguity and make a judgment call on whether the outcome is good or bad. Typically, middle or junior managers find those kinds of judgments difficult, so they default to measuring time because that's easy.

How do you know AI is working for your organization without using simple time saved measurements?

Coplin: I think you've got to listen to your people and listen to your customers. Those measures help you avoid some of the problems that we're experiencing with AI. People who are using AI to replace what they do rather than to elevate it. These are the people who will copy and paste the answer to a prompt without checking it or putting any effort into adding their own personal context. Yes, they can generate a lot, but is the customer delighted by it? Have teammates been helped by it?

Back to that subjectiveness about quality and about whether this materially moves. Does this materially give me what I was looking for as a customer? I think at the point at which you're measuring that, the other sort of by-product of that is we'll start to care a bit less whether AI was used or not.

I'll give someone a deliverable, and they'll ask whether I used AI to write it. And if I say, 'Yeah, I did,' their reaction is most likely not favorable, based on perception. What I should be saying is 'I did use AI, and I didn't get it to write it for me; it wrote it with me.'

I probably spent more time gathering all the context and developing it than I would have if I'd written it myself. So, I think those are the measures of success. It's about quality and effectiveness. The measure of the 21st century needs to be effectiveness. So, are my customers delighted with it? Have I empowered my teammates to delight their customers because of what I've done?

I think the single biggest challenge facing managers today is equipping them to measure outcomes. The problem with outcome versus time is that time is tangible, while outcomes are subjective.

Describe the differences between effectiveness and efficiency in business, and why this should be the business guiding principle.

Coplin: We've been running IT efficiency programs for decades. And essentially, we do the same stuff a bit quicker and a bit cheaper -- whether it's moving to the cloud or what we're doing with AI. Let's use the same processes but do them more quickly and cheaply, because we'll be more efficient.

Effectiveness is not about whether I can do the same things more quickly or more cheaply. It's whether I'm doing the right things. It's a different focus.

I can be making the widgets, but if the car coming off the production line is rubbish, then actually, are we being successful as an organization? And it requires people within the organization, particularly leaders, to think about what we do.

I have an example in the UK. I'm on the board of one of the UK's largest hospitality companies. We run lots of national chains. As the chairman, we have this sort of running joke: we're not in the hospitality business; we're in the entertainment business. Because the product of what we do is not the beer or food we serve, that's where the traditional focus is. It's the experience a customer has at one of our facilities while consuming our beer or food.

In my opinion, this highlights the difference between efficiency and effectiveness. Because if we focus on the beer and food and how to get them cheaper and better, we will miss the importance of the environment we're creating and the customer service the consumer receives. At the end of the day, the customer is choosing to spend more money than they would have if they stayed home. They're going to be consuming a product they might have had at home, just in a nicer environment. So, it becomes all about the experience and what they do.

This pivots into effectiveness. I think we should start measuring our AI transformation programs by effectiveness.

How do you address the fear people have of losing their jobs to AI? How do you think this will affect the progression of AI?

Coplin: I have this conversation with my clients, who are typically not technical people. They are the CEOs or the C-suite of big organizations. And they're starting to ask, should we be doing this with AI?

I'm advocating that we spend a bit more time looking for the appropriate use of AI. So, not to replace the skills, not to scoop out the bottom of the career ladder from our young people who are graduating, but to show what happens when you have human plus machine rather than humans vs. machine. And I think as a society, we would do better to have a bit more of that dialogue.

I do think business leaders need some help on two levels. Number one, it's a generalization, but I found it to be true anecdotally. Most people I know use AI like a glorified Google search, give it no context, don't provide enough information and are probably asking the wrong question. And as a result, they think it's OK, but it's not that great. So, we're starting from the wrong position. Secondly, we're not really showing them how to elevate the organization's capability. So, AI to me isn't about doing more with less. It's about doing more because we've created the capacity to deliver more value.

IKEA moved to AI, except rather than getting rid of all of their people for the mechanical processes AI could do in the factory, IKEA said, 'We don't need you here, but we'd love you on the shop floor, and we'll retrain you as interior design consultants to work with our customers.'

To me, that speaks to the culture of an organization and to the future I want -- where organizations move people from running factory machines to where they are needed with proper retraining.

You said shadow AI is a good indicator that people want to change and be proactive. How do you address this without impeding the AI excitement? And, how do you manage the risk associated with shadow AI?

Coplin: There's a subtext to the problem: the more prevalent technology becomes in businesses, the greater the perceived risk, and businesses ask IT to manage it. And in doing so, the industry is building many walls, rules and governance to keep the organization safe, which are now kind of getting in the way.

Now, many people are trying to straddle both sides. They're trying to embrace risk while also enabling innovation. I think there's still a reasonable proportion of people who are just stuck inside the risk and don't want to change. So shadow AI is a good signal that your people want more from technology.

I've spent three decades in IT, and most of that time was spent trying to get people just to care even a little bit about what technology could do. Now with technology, the business units are saying, 'This is amazing, help us do it more and take us further, and we've got some great ideas.'

I fear that if we treat that with governance and risk, and we try to turn it off and shut them down, we'll lose them. And they'll go and do it anyway because they can, because it's so much easier to do for themselves now.

It just puts you in a place to harness that energy and the goodwill. Part of my argument is that what we've lost with IT and the business is our relationship. We do have a barrier between us, and we are almost serving each other rather than being peers.

I think this is the opportunity to reframe that because the business knows what it wants to do better than we do in IT. But at the same time, IT has a better idea of where technology can go and what it can do that business units don't understand. And it's the marrying of those two things together. So, I'm trying to encourage IT leaders to embrace shadow AI, along with the risks it entails and to find an appropriate path forward to ensure we harness the energy and momentum we have been crying out for years.

It's not a free world where business units can do whatever they like; it's about figuring out the most appropriate way forward.

How do you empower and encourage businesses to take control of the risk beyond IT demands? Any other recommendations for IT to work with the business and navigate AI initiatives to drive business opportunities?

Coplin: Technology is no longer the thing that only happens in the IT room. It happens throughout the business. I think it's helping the business to see that the technology risk is also a business risk. Under that shared ownership, we can make better decisions.

Managing risk articulates in a way that the business understands well enough to decide whether it's okay to do that. Cybersecurity keeps the CFO and the CEO up at night because it's the one thing on their horizon that they know they have zero control over. And it's such an alien concept for any CEO to know that there's a problem they have no power over, whether it will happen or not. Of course, they can manage the risk. And of course, they can improve, such as avoiding outages.

But I think these are important conversations. It's meeting with the board and saying, 'Look, I can't give you a 100% guarantee, but how much confidence do I need to give you? Is it 60%? Is it 80%? Is it 99%?' Then, figure out the best way to handle the risk.

In summary, it's a much more mature conversation about risk, but more importantly, it's a conversation that's across the business rather than just stuck in the IT department reporting back to the board.

Amanda Hetler is a senior editor and writer for Informa TechTarget, covering IT strategy and CIO topics, including AI, security and risk management, and digital transformation.