Accelerating AI adoption and governance amid an AI slowdown

Despite calls for an AI slowdown from leaders of AI frontier labs, most enterprises are more focused on strengthening their governance than on slowing their use of AI.

A new report from OneTrust, an AI data privacy and security vendor, found that while many enterprises experienced AI-related problems in the past year, most are not slowing or pausing their AI deployments. Those incidents include IP exposure and AI agents using and delivering and deleting data.

In this Q&A, OneTrust's chief innovation officer, Blake Brannon, discusses why enterprises continue to push forward with AI and how they can focus even more on governance than on the rapid advancement of AI models.

The report mentioned that despite the AI incidents, enterprises are still encouraging the use of AI and AI agents. What is fueling the forward momentum to deploy the technology?

Blake Brannon: From the top down at every company, the number one topic your board is asking you is ‘What are we doing with AI? How are we transforming the way we work, the products and services we build? Are we going to be disrupted by the frontier models, by incumbent startups? What is our moat?’

The pressure is causing everyone to be a little aggressive, accelerating and figuring out how to use AI.

Once you start looking at tools, using tools, and trying to deploy these tools, you very quickly start to worry about how you are going to govern it. How you are going to secure it. Everyone is looking at the reality that there are going to be more agents acting than there are humans in every company in the world, within the next two years.

The programs and processes they have built over time to govern the company were designed in an era when a human instantiated everything the company did. Either a human was doing it, or a human was building a deterministic software system that was doing it. And that allowed you to be able to govern those things because they were moving at that pace. Today, you have the citizen developers, the citizen builders. Everybody can build an agent.
That fundamentally breaks down how traditional governance and security have worked in organizations, because whatever organizations did to protect and safeguard their data used to take weeks. You must figure out how to do that in seconds now.

How does this change lead organizations to invest in governance? How can vendors help with this?

Brannon: You can create all this great AI, but if you do not trust it, you cannot turn it loose.

You cannot let it run autonomously, so you must solve this. It is becoming critical that enterprises are now ready to invest because people are moving beyond proof of concepts. They are getting to scale. With scale, you have this simple question of how to trust, know what is happening with and control AI. That is why organizations are investing more time in this. One way I like to think about framing is that you must have your AI systems behave the way you or one of your employees would. You must get it aligned to the way you think. What you, as an organization, effectively says is okay  to do as an action is tied to and rooted in factors like compliance, your security practices and your brand promise. You must mimic that with these agents. That is one part of the problem.

The second part of the problem is that you need to prevent bad things from happening. So, you must have the car's emergency brakes on. You must see that this agent is going to take destructive action, and you want to stop it. And that is a very nuanced thing because AI systems do not inherit things like people's identity and permission. We want humans to be in the middle of something like that.

As organizations are moving fast and using AI, how do they keep up with governance as the technology and vendors update their models so fast?

Brannon: Two principles are shaping the architecture to allow this trust to be associated with these systems. The first is that we see someone post a blog or article almost every weekend saying, ‘Please regulate us, we can't trust our systems, and this is very dangerous.’

What that means is that you cannot trust a model. You must assume a model will break out of its harness, and a model could manipulate itself to convince itself that it is okay to do anything it is doing.

The governing architecture must be an independent, separate harness from the actual thing you are governing. This is how society has worked for thousands of years. Government power is separated intentionally. Company departments are intentionally separated to create a clear, bias-free distinction between the governing body and the thing being governed.

What that means is that the models and the providers need an independent governing agent, or a governing harness, to assess what that AI system and agent are doing.

The second emerging principle is the complexity of keeping up with the models and the harnesses.

For any organization, it will be a fragmented set of different model providers, models, and harnesses that they are all using because marketing wants to use something, customer support needs to use something else, and engineering needs to use something else. Everyone will have a very fragmented set of these things, which creates tremendous complexity in how you control and govern these models when you have a whole bunch to deal with, and they all behave differently.

The architecture is shifting to say, ‘Who cares? Who cares about what the model is doing? Who cares why the model thought what it thought?’ What I care about as an enterprise or an organization is when that AI system actually goes and takes an action, when it tries to read data from an enterprise system, when it tries to send an email, or when it tries to delete a record.

That is when I care about governing something. I can design my governing controls around a fixed point where it's connecting to my stuff I care about, and to be able to put the right guardrails and policies in place on that fixed point.

This Q&A has been edited for style and conciseness.

Esther Shittu is an Informa TechTarget news writer and podcast host covering artificial intelligence software and systems.