The fall guy: CIOs get blamed for AI errors

Transcript - The fall guy: CIOs get blamed for AI errors

Kelsey Sung: Who gets held accountable for AI errors?

It's a question that has been haunting enterprise leaders as they try to transition their businesses into the AI era. And it's the question we plan to dive into today with Paige Gross, a reporter from CIO Dive.

Hi, Paige.

Paige Gross: Hi, Kelsey. Thanks for having me.

Sung: Yeah, it's a pleasure.

So, give us the breakdown of this 8x8 report that kind of covers the question that I mentioned at the top of the episode. Tell us who did the survey, who were they surveying and what did they find?

Gross: Yeah. So, for this survey, they talked to 2,500 tech executives in the U.S., U.K., France, Australia and Ireland over the course of July. You know, 'tech executive' can really be a couple of roles. It could be CIO, CTO -- and it can actually change quite a bit depending on the organization. But, basically, they found that, in who was to blame when something went wrong with an AI output or got a result that kind of made it farther than it should have, CIOs have ultimately taken that responsibility, that blame. It was only 16% of customer service and 6% of legal leadership that, you know, followed those tech executives in the blame game, sort of.

And the report really highlighted that fragmented governance and the documentation process can kind of contribute to this lack of control that we're seeing. The CEO told me in an email that there needs to be more structural guarantees that CIOs can actually see what AI did before, during or after an incident.

Sung: Interesting. So, I'm curious how these findings fit in with what you're seeing in your reporting about the CIO role in enterprise AI. Obviously, there's a lot of things we could dig into here, but I'm very curious about that sort of blame that's being associated with the CIO and if that's something that you've already been seeing or noticing.

Gross: Yeah. I mean, I think AI is changing so much about how enterprises operate. I'm sure business reporters everywhere are seeing this on all levels and whatever their beat is. But my team, particularly, has really been reporting on sort of the expansion of the CIO role. It has really changed from just kind of managing the technology and sort of being like the head of IT to really being someone who oversees organizational changes throughout the enterprise. You know, essentially making sure that these AI transformations kind of actually get embedded into workflows. They might be managing budgets, and now we're hearing, you know, they're taking the blame when things go wrong.

So, the CIO role is really expanding as a part of just the overall AI transformation. They're now responsible for all of the ways that AI touches an organization, more so than just kind of running the IT team -- which, maybe 10 years ago, that was probably their primary goal.

Sung: That's interesting. So, you kind of touched on this before, but one of the major things that your article highlights is that checking an AI's answer isn't enough. You actually have to kind of reconstruct how to reach that answer, and that can require a lot of visibility, a lot of documentation, and there's some gaps there for CIOs.

What visibility and documentation are CIOs generally missing? And what was the cause of that gap between their accountability and what they actually can control, according to the research that you were looking at?

Gross: You know, AI ownership -- or, when I say ownership, it really boils down to, like, who's going to take responsibility or who has the authority to manage AI. You know, this has been a pain point for organizations as long as they've been deploying the technology. Most CIOs don't actually have the team-level visibility into how AI is being used. So, they might sign off on a tool being used, you know, across the organization. But they, on the day-to-day, don't really see how accounting is using it versus how HR might be using it.

And I think a lot of what I cover lately is about governance, and governance is really like the structural controls and documentation that you put in place around AI. And a lot of my reporting has shown that enterprises are really deploying AI before their governance is really there. Some feel like maybe just a little bit is OK; they're kind of experimenting with how much controls really make for safe AI use. But it's really hard to keep track of what went wrong if there isn't that system of documentation and if there isn't, like, a chain of command when it comes to an AI mistake or an AI output.

So, I think a thing that my team and I talk about a lot is, like, when everyone shares the responsibility of AI, no one really does. And I think that's sort of what a lot of tech executives are finding, and a lot of the middle managers who are actually the ones encouraging their teams to use it, and even the employees themselves who, you know, are just kind of getting this top-down instruction to use it -- is that if the documentation and the governance aren't there, it can often feel like this, right? Where no one really knows what the right output we're looking for, no one knows how to document it, no one really knows how to say when something goes right, too. Right?

So, I think all of the documentation -- as, like, unfun as that sounds -- just like any kind of transformation within an organization, it really is the bread and butter of how to find success in these AI deployments.

Sung: Something that comes to mind for me is, are we right to place so much responsibility in the CIO specifically? Like, is this the role that the CIO is built for? Or is this something that you think is misplaced? I'm sure there's a lot of conversation going on around that. I'm curious about your opinion, but also if you don't want to give that, that's OK.

Gross: Yeah, no, that's fair. I actually have seen, in the last maybe two years, more prevalence in a specific AI leadership role emerging. So, you have, like, you might have a CIO. But you're also seeing, like, a chief AI officer joining the suite. We've covered that a bit over the last few months. We've seen more of that entering the fold because, I think, people are realizing what a responsibility it is to -- you know, a CIO is also, you know, thinking about a lot of other things outside of just AI, right? Like, enterprises use a ton of SaaS products, just general software. Essentially, like, the cybersecurity team could sit under there. So, like, AI, though it's a lot of what we talk about and think about because it's so new and that everyone wants to use it, it is a small part of what an overall tech team these days needs to be thinking about.

And so I think that, no, it's maybe not fair. I don't know that the blame is rightfully placed totally on a CIO. However, it's kind of one of the things that sits under the chief innovation officer. It makes sense, and it doesn't at the same time. So, I see how and why some enterprises are pushing forward that chief AI officer role to say, like, we realize that this is becoming a huge part of what we're responsible for and not necessarily one person can manage and be responsible for.

AI is now really touching so many parts of an organization. We've been covering how it's making members of the C-suite -- like, a CFO and a CIO are actually now working quite closely together because the budget of AI is becoming exponential. And, like, token cost is something that people have to think about. And so, it's not just changing the role of the CIO, but it's really making the whole C-suite have to talk more amongst themselves, have to talk more about budget, kind of get approvals through things. And so, I think it makes sense that this newer role would be added to the fold to manage all of this.

Sung: Yeah, it's like it makes sense that the blame, the attention has settled on this CIO role, but for the long term, that might not be fair to put.

So, what would need to change going forward for the CIO role? Like, assuming that maybe a company isn't in a place yet to evolve this new type of role for AI specifically. How do we make this role more fair for the CIO?

Gross: It was kismet that we're talking about this today because the story that I'll be publishing for our newsletter tomorrow was of a similar vein -- realizing, basically, that AI use and decision-making about AI kind of happens all across the organization. You have, like, heads of departments making decisions about it, but the report came to the same conclusion that CIOs are often the ones holding the blame when something goes wrong.

And they, that report -- I'll have to tease the headline when it's actually worked out with my editor -- but the report basically suggests that CIOs should lead, essentially, a rinse-and-repeat-type process, where they establish responsibility and accountability within departments. So, let's say, like, if HR is using it, they would kind of create a structure around how to document, you know, misuses or mistakes with AI. And they would sort of create, like, a responsibility documentation process that can be repeated throughout the organization -- sort of recognizing, like, a lot of this is out of my hands, but we could create the structure for responsible AI use and mistakes, and we could kind of, like, as the CIO, my responsibility would be implementing that process. Maybe not overseeing every little thing that happens, but creating systems within your organization that can ultimately lead to more documentation, more visibility, stronger governance, etc.

Sung: Yeah. Yeah, I mean, it is a tall order to try to have an eye on what everyone in an organization is using in terms of AI and what practices they're following or not following.

Anything else to add that we might not have touched on? Or anything that we did touch on that you want to dig into a little deeper before we get off?

Gross: Yeah, I mean, I think AI just changes so quickly. I feel like it's transforming at a rate that most business technologies have not, right? Like, I've only actually been with CIO Dive since March, and I've seen whole trend cycles exist within enterprise AI in the last, like -- what is that? Six, seven months? And so, just to say, like, the transformations and the changes that we're seeing with, like, the CIO role and how they handle it now could just even by the end of the year be totally different. CIOs who, I guess, feel like they are struggling to manage it -- the reports all show you're kind of all in the same boat.

Sung: Yeah, at least there's solidarity.

Gross: Yeah, and that it's probably going to continue to change. So, it's great to have those foundations about governance, accountability, how to kind of do audits on your AI use cases. But it will continue to change, and we could be back here in a few more months with totally different answers.

Sung: Very true. We've seen that happening all throughout, you know, the last couple of years.

So, well, thank you, Paige.

Gross: Thanks.

Sung: And for our listeners, if you're interested in hearing more from Paige, check out her work on CIO Dive and follow her on LinkedIn. See you next week.