Organizations are deploying AI agents at unprecedented scale. Gartner found that while the average Fortune 500 enterprise used fewer than 15 agents in 2025, that number will balloon to more than 150,000 by 2028.
Given recent incidents of AI agents escaping their testing environments, executives and board members are rightfully demanding accountability and security for the AI agents deployed in their organizations. Human-in-the-loop is a common default response, but bottlenecks, latency, scalability challenges, poor feedback loops, and human decision-making issues and fatigue are common arguments against it.
"If a person's reviewing hundreds of decisions after the fact, or if they're approving actions they can't independently validate, or they don't have the expertise to know what they're looking at, is that oversight or is that just some sort of accountability theater?" asked Jess Burn, an analyst at Forrester Research, during a panel session at Black Hat USA 2026.
If you don't give agents agency, then what are you doing?
Will Pearce, co-founder, Dreadnode
"Human-in-the-loop is one of those things that people like to say," said Nathan Hamiel, senior director of research at Kudelski Security. "You want someone to blame when something goes wrong."
The case for letting agents fail
Will Pearce, co-founder of Dreadnode, an AI-native security company, advocates for letting agents do their thing -- within reason.
Regulated industries must follow specific guidelines, he noted. But if the average SOC has a handle on agent permissions and guardrails, it can and should "explore the edges of what's possible," permitting AI agents to run and, yes, fail. He cited the use of red teaming, pen testing and vulnerability scanning -- all conducted to find and fix weaknesses. AI agents should be treated no differently.
"If you don't give agents agency, then what are you doing?" Pearce said. "The whole point is to scale beyond what you could before. If you constrain it to this human-in-the-loop -- it's really expensive not to let it go. In those failures you're finding moments where something wasn't the way you thought it was."
Pearce added that when deploying AI agents, "there's a step that defenders need to take where it's not caring less, it's just becoming less policy-driven and more action-driven."
Of course, this won't work with every single scenario, Hamiel said. "The problem -- as with anything that you apply AI to -- is that everything is a risk-based decision."
He explained that the success of human-in-the-loop initiatives largely depends on the organization's understanding of its agent use cases, and its risk profile and appetite for potential failure. As risk scales, so will the need for human-in-the-loop.
Victoria Westeroff, principal of AI red team ops at Microsoft, agreed. "Knowing where really big decisions are in the process and the tech stack should be the guiding principle of where you're mandating human review," she said.
Building secure agentic systems
Several factors go into securing AI agents. "You can't treat [an agent] like a traditional software component," Hamiel said, adding that in terms of access, an agent acts almost like a human hacker. "Not to anthropomorphize it, but you're giving it access to tools, you're giving it access to files, you're giving it access to things it needs to be able to perform an action."
In terms of access control, the experts recommended providing agents only with the access and permissions they need, following the principles of least privilege, least access and least agency.
To do this properly, teams must understand the agentic workload and its mission. This includes knowing agents' intentionality, Westeroff said.
"If you start building agentic ecosystems without really clear expectations about what they're solving, it's really difficult to know if they're going out of lane," Westeroff said. "To actually monitor and call those agents out when we see behavior that's not intentional requires you to have definitions of how you want those agents behaving and pretty strict personas or lanes. You really need to know what you want those agents to be doing."
This involves intentionally integrating agents into workflows, knowing stop gaps and understanding where decisions can be autonomously pulled back -- if possible.
Guardrails and data complexity
Once teams understand intentionality and definitions, they can better classify what constitutes a "good" agent across various scenarios. And when failures occur, they can learn from them and put security guardrails in place.
"There are all kinds of different experiments you can run," Pearce said. "As long as you have the ability to collect that data, replay that data and observe what's happening, you have the ability to write a guardrail for next time."
Admittedly, Pearce said, keeping up with those guardrails is hard. "It's not this one-off thing. You need systems that can hold data, you need to query it, you need to sandbox -- there are a lot of different technologies," he said. "Finding these issues isn't hard; it's just the continuity of the improvement that is difficult to manage."
Beyond a scaling continuous improvement process, human-in-the-loop requires diagnosing issues. People must understand the data an agent provides and its context -- an area where current capabilities often fall short.
"If you're in a multi-agentic system, you need to understand actions across multiple agentic actions and actors. The data munching gets really, really complex," Westeroff said. "If you don't have the tools or folks who know how to do it, then the human-in-the-loop is inherently not going to be the expert needed."
As the number of agents is expected to scale in the months and years ahead, this will become exponentially more difficult.
"You need to invest in a way to map the data that you get out of agentic ecosystems to something that the expert decision-maker can actually compute quickly and decisively -- and I think we have some work to do," Westeroff said.
Sharon Shea is executive editor of TechTarget Cybersecurity.