Organizations struggle to balance AI autonomy with real human oversight, as human-in-the-loop is often symbolic rather than substantive, risking cognitive atrophy and rubber-stamping.
As AI agents increasingly take on more work, organizations are struggling to find the right balance of human and AI relationships.
Central to this are the concepts of both human-in-the-loop and human-on-the-loop, in which a human worker is an integral part of the process that employs an AI agent working autonomously. But there are serious questions about what human-in-the-loop really means and whether organizations are deploying these models in a way that takes the best advantage of what both humans and AI agents can do.
There's no question that the use of AI agents to perform real work in organizations is on the rise. The 2026 State of AI in the Enterprise report by Deloitte shows that 74% of companies plan to deploy AI agents within two years. However, this influx of agents may be outpacing the ability to manage them. In the report, just 21% of survey respondents said they have a mature model for governing AI agents.
The answer for many is to implement a human-the-loop strategy to oversee AI agents. But there are questions about whether organizations are integrating the human-in-the-loop effectively and whether their designs are optimized to get the most value from AI agents. Getting this wrong can have serious consequences for organizations.
Give humans authority
Human-in-the-loop is essentially a meaningless phrase, said Dan Leiva, founder of CXAmplify, an AI executive advisory firm.
People in companies say they are putting a human-in-the-loop, but they don't really mean it because they don't give them the right agency and authority, Leiva said.
"They say 'human-in-the-loop' the same way they say 'sign this form,' and no one reads the form. It's just a meaningless phrase," he said.
But there are three classes of decision-making that AI should handle, and a human-in-the-loop may be required to supervise the AI agents, Leiva said.
The first class of AI decisions is in cut-and-dried processes such as package routing for delivery companies, where the AI agent can make decisions based on specific rules. Second is a situation in which some human judgment may be required; for example, when an AI agent flags a shipping return exception and a person reviews the exception. Third is a regulatory, ethical or legal situation in which a human must make the final decision.
Once companies know how to structure these human-in-the-loop processes, they need to name an actual human who's responsible for the decisions, he said.
"Agency and authority mean that there needs to be someone who's responsible for [the decision] and who has the authority to say that something's wrong and we should push the red button and look at it," Leiva said. "You have to do that without penalty, because if you give it a penalty, no one's going to push the red button. Human-in-the-loop has to have the authority to say, 'I think it's wrong.'"
AI agent use may lead to cognitive atrophy
Dr. Fern Halper, industry analyst and founder of AI Foundations Group, acknowledged that enterprises want to include mechanisms for human judgment over AI agents, especially when they are beginning to make influencing decisions.
But Halper has a skeptical view of the value of having the human-in-the-loop or human-on-the-loop. Many organizations are using human-in-the-loop as a reassuring phrase without fully defining the human role.
"Is the human actively reasoning and participating in the work or are they simply reviewing outputs and then clicking approved, because those are two very different governance and cognitive models," Halper said.
Is the human actively reasoning and participating in the work or are they simply reviewing outputs and then clicking approved, because those are two very different governance and cognitive models.
Dr. Fern HalperFounder, AI Foundations Group
There are a few ways that human-in-the-loop is manifesting itself, as organizations move to implement more autonomous AI agents, she said. On one end of the spectrum, humans are merely validators of the work agents generate. At the other end of the spectrum, AI is helping expand human capabilities in areas such as cancer research.
"But that's a very different cognitive relationship than what's happening when people are just checking over what the AI is doing," Halper said.
Research also suggests that delegating work to AI agents may be reducing critical thinking in humans, she said. This results in cognitive atrophy, where the humans lose knowledge of the very processes that they are tasked with validating.
For example, AI agents are increasingly used in call centers, and the human-in-the-loop is often engaged when the AI agent can't handle an exception, which then gets passed along to the human. Because much of the work of junior professionals is being handed over to AI agents, the humans-in-the-loop are not acquiring the knowledge of the processes needed to resolve complex exceptions.
The human-in-the-loop is also leading humans to simply rubber-stamping whatever the autonomous agent sends through, Halper said.
AI has great potential, but enterprises aren't thinking about issues such as cognitive atrophy and what human-in-the-loop actually means, she said. It may also lead to automation bias, where people simply accept the results of anything that's fed to them by automation.
"A lot of times it's just saying, 'Can you review this,' but it's reviewing a bunch of AI-produced work slop," Halper said. "That's not cognitively engaging at all. People just believe what comes out, and they're willing to trust it. That's a problem."
Effective use of AI agents and humans
There are areas where human-in-the-loop processes are becoming a necessary part of enterprise operations.
Babak Hodjat, chief AI officer at Cognizant, said that the company has introduced AI agents for internal processes.
The agents are supervised by human workers in both human-in-the-loop and human-on-the-loop situations, said Hodjat.
For Cognizant's AI agents, human-in-the-loop means that humans must sign off on any transaction that the agent initiates, based on criteria such as legal review, regulations or ethics, he said. The human approval requirement is built into the agents' design.
Cognizant is introducing more human-on-the-loop agent processes as trust in the AI agents' decisions increases, Hodjat said. Agents will make more decisions autonomously, but humans can be brought into situations that require review.
"As we trust the system more, we want to delegate more and have the agents do that autonomously," Hodjat said. "We have a number of ways to surface the transaction to have a human take a look at it."
For example, the system can flag a transaction that's out of scope or is in an out-of-context scenario, he said. In many cases, this means that agents are supervising other agents.
"[We might have] an agent looking at the work of another agent and saying, 'For this particular transaction, a human must come in,'" Hodjat said.
Watching a child genius
The human-in-the-loop is integral for reining in AI agents, said Mike Kazmier, head of AI at Banyan Software, a private holding company that acquires, builds and manages enterprise software businesses.
Human-in-the-loop is very prevalent and embedded in Banyan's processes, he said. But there's zero chance that enterprises will let AI run completely autonomously, as AI agents are akin to highly intelligent 5-year-old children.
"They're incredibly capable and can do amazing things, but every once in a while, they'll bolt to the left and run in front of a car and do things that you don't want a 5-year-old to do," Kazmier said. "So having the right guardrails in place and the right process to both catch and correct that behavior in the future is super important."
Banyan is designing AI agents to mimic the same human processes it uses, allowing team members to become orchestrators, he said. It also enables Banyan to systematically assess the agents' outputs.
"We don't try to reimagine processes completely on day one," Kazmier said. "First we automate, then we embed AI and follow similar processes, then we refine after we're comfortable with the impact that we're seeing."
Banyan is continuously building in observability, security and risk reviews of those processes, but human control is still at the center of getting the value from the AI agents, he said. But they will not take away human labor.
"We look at AI as just a tool, not a replacement right now," Kazmier said. "It's about how you wield that tool to create leverage for yourself within the same core processes where the person is still responsible for the output of that AI capability."
Jim O'Donnell is a news director for TechTarget, where he covers IT strategy and enterprise ESG.
