“Last year was all about building agents, this year it’s all about scaling them,” according to Dinesh Nirmal, IBM’s senior vice president of products for IBM Software. Omdia research has shown that enterprises are struggling to move from proof of concept to production. This struggle shows in two key numbers:
- Only 22% of enterprises use at least some agents in mission-critical production scenarios (Omdia research, 2026).
- The average enterprise takes approximately 12 months to move a successful PoC into production (Omdia research, 2026).
This shows that enterprises are in a race to figure out how to benefit from AI at scale. IBM correctly determined that AI model performance itself is no longer the bottleneck. Now, enterprises need to figure out how to overcome the foundational shortcoming of LLMs that lies in their probabilistic output.
You can think of LLMs as massive pattern-matching machines that build a sentence one term at a time, each time choosing the term most likely to follow the ones already written. This illustrates that LLMs do not have an understanding of how the world actually works and are therefore prone to mistakes that humans would never make.
For example, ask a model whether a guest landing at 20:41 will make the 20:30 airport shuttle, and it will correctly answer "no" and rebook her onto the next one. What it will not do is ask whether the 20:30 shuttle actually left at 20:30. Nobody told it the shuttle was stuck in traffic, so it assumed the schedule was true because the schedules it has read typically were. The guest reaches the curb, the 20:30 shuttle is still there, and her seat on it has been given away.
The key to agent scalability
Agents that come up with innovative new ways of upsetting customers — for example, by giving away their spot on an airport shuttle — need too much babysitting when adopted at scale. To rule out these seemingly trivial mistakes, enterprises need a framework that provides LLMs with the fresh context and guardrails to consistently achieve deterministic and therefore actionable results. Once agents are able to translate complex context scenarios into clear, transparent and repeatable actions, enterprises will hold the keys to harnessing agents at scale without having to fear negative repercussions.
Even 99.9% repeatability of accurate results might not be sufficient. For example, Nirmal described an IBM customer whose agents make 8.5 million model calls per day. At 99.9% accuracy, which is better than production LLMs can deliver, this results in 8,500 wrong answers a day and about 3.1 million a year. The only way to catch and correct these mistakes is a deterministic layer around the model consisting of fixed inputs the agent must read before making a decision, fixed checks before it acts and a log of every call so the 8,500 can be found and reversed.
Let’s take a simplified example from the hospitality industry: Rebooking one delayed guest to a later airport shuttle takes five steps: read the new arrival time, check the next shuttle, change the pickup time, hold the room and decide whether to charge a late fee. If each step is 97% reliable, the chain is 0.97 to the fifth power, or 86%. Across 1,000 delayed guests, 141 get a wrong outcome: a missed shuttle, a released room, a fee they should not have paid. The agent does not need a better model to fix this. It needs the five inputs delivered as up-to-date facts rather than guesses, and a check before each step so most of the 141 never happen.
At scale, the question is never whether the model is good enough. It is whether the mistakes it will make — whether that is 0.1% or 3% of its decisions — can be prevented where possible, found where not and undone in every case. Deterministic context and audit trails exist for exactly that, which is why IBM spent the keynote on data, identity and runtime controls rather than on models.
IBM’s grand vision: 4 layers around the model
The model still does the reasoning. What changed at TechXchange 2026 is that IBM now surrounds that reasoning with four layers that each take a type of decision away from the model: what facts the agent sees, which functions it can call, what it is allowed to do with them and what gets recorded afterward. Here they are in the order an agent meets them:
- Layer 1: The facts the agent sees (Streamhouse and Infragraph).
- Layer 2: The functions the agent can call (Integration Intelligence Hub).
- Layer 3: What the agent is allowed to do (Agent Identity, Vault, Watsonx Orchestrate, Terraform policy).
- Layer 4: What gets recorded (Watsonx Orchestrate and IBM Concert).
Layer 1: The facts the agent sees
Most of the 141 wrong rebookings start the same way: The model reasons correctly over an arrival time or a shuttle schedule that was already out of date when it read it. That is why the first product IBM put on stage was not a model or an agent but a way of keeping the facts current. Enter Streamhouse.
Streamhouse is a data architecture, not a product. It captures each change in any enterprise data source as it happens, joins and transforms those changes while they are still in motion, applies governance rules on the way through and serves the resulting current state to the model at decision time. IBM’s implementation builds it from four open source components:
- Kafka carries every change out of the source systems the moment it happens, so the reservation update and the new arrival time are on the wire within a second.
- Flink joins and transforms those changes while they are still in motion, turning the separate events into one record per guest, shuttle or order.
- Iceberg stores the result as a table that is always current and that reports and lakehouses can read as well.
- Confluent’s Real-Time Context Engine serves that table to the agent over an API or Model Context Protocol (MCP), so the agent reads one row instead of querying four systems.
The agent does not go looking for a flight delay. The airline’s arrival feed publishes it, and within a second, the streaming layer has joined it with everything else the hotel already subscribes to: the shuttle’s live position, the current customs wait at the airport and traffic on the road to the hotel. From those, it computes two numbers per guest — when that guest will reach the curb and when the shuttle will actually leave — and stores the difference. The agent reads that one row.
For Guest A, who's on a slightly delayed flight and whose shuttle is itself running late in traffic, the difference is positive; the agent does nothing, which is the right answer and one a human would have missed while rebooking everyone whose flight showed a delay. For Guest B, who lands 49 minutes late into a 25-minute customs queue, the difference is negative; the agent moves the pickup, holds the room and waives the late fee if the loyalty tier allows it. At no point does the agent poll a system, estimate a wait or add minutes to a timestamp. The arithmetic is done once, deterministically, on current data (that’s what the Streamhouse architecture does), and the model’s only job is the decision at the end.
Infrastructure context in near real time (HashiCorp Infragraph)
HashiCorp Infragraph does for infrastructure what Streamhouse does for business data. It holds a graph of cloud resources, the Terraform state that created them and the dependencies between them. An agent reads from the graph what a change will touch, which systems depend on it and what will break if it goes wrong — all before it acts. A site reliability engineering (SRE) agent asked to tighten a security group sees in the graph that the same group also fronts the payments API and two batch jobs. From there, it either widens the change request to cover all three or declines, instead of finding the dependency when the batch jobs fail at midnight.
Near real time is the right qualifier. Infragraph refreshes on a schedule, daily by default, whereas Streamhouse updates as events happen. For the security group example, that is fine; a dependency between a security group and a payments API does not change from one hour to the next. It is not fine for an agent that needs to know whether a resource was modified five minutes ago. IBM said in the analyst Q&A that running Infragraph’s data through Streamhouse to close that gap is in progress, so for now the two layers deliver the same kind of context at different ages.
Together, Streamhouse and Infragraph give the agent one place to read the current state of the business and one place to read the current state of the infrastructure. That is the first condition for a repeatable decision: Every agent works from the same facts, rather than each assembling its own picture from six systems on every transaction and getting a slightly different answer.
Layer 2: The functions the agent can call
Once the agent knows Guest B will miss the shuttle, it has to move her pickup, hold her room and check her loyalty tier. Each of those is a call into a system the hotel already runs. An agent that works out how to make those calls from scratch on every transaction is slow, spends tokens solving a problem the integration team solved years ago, and gets it slightly wrong often enough to matter.
IBM Integration Intelligence Hub removes that work from the model. It catalogs the APIs, workflows and queues the enterprise already operates — including the ones built on WebMethods, MQ and API Connect — and knows which depend on which. When an agent asks how to move a pickup, the Hub delivers the existing “change transport booking” workflow into the agent’s context at that moment, with the policy that governs it, rather than every integration it has ever seen. The agent calls a function someone already built and tested instead of composing one.
The gain is twofold: Decisions become more accurate because the agent chooses among defined actions rather than inventing them, and token costs drop because the integration logic is not regenerated per session. The keynote demo showed this with a loyalty lookup the agent could not see until the Hub exposed it: The function existed, but the agent just had no way to find it.
On the infrastructure side, Terraform modules, the Terraform Developer Portal and the Terraform MCP server do the same job for provisioning. The agent requests a change through a module the platform team published, with the policy attached, rather than writing its own HCL. Determinism at this layer means the action the agent takes is one a person already defined.
Layer 3: What the agent is allowed to do
Current facts and prebuilt functions remove most of the reasons an agent gets a rebooking wrong. They do not remove all of them. The model can still read the right row, pick the right function and apply it to the wrong guest. Layer 3 is the set of rules that stop a wrong action before it executes, whatever the model’s reasoning was.
IBM Agent Identity gives each agent its own verified identity, separate from any human user, and HashiCorp Vault issues it short-lived credentials scoped to the task at hand. The rebooking agent for Guest B gets a credential that can move her pickup and hold her room — and nothing else — for the next 30 minutes. If the model decides mid-task that the easiest way to free a seat is to bump Guest A instead, the call fails on the credential before it reaches the transport system, and the session is flagged. That was the live demo in the keynote: The agent was asked to change a different guest’s reservation, was refused, and the session was cut. The refusal is a rule, not a model judgment, and it fires regardless of how confident the model was.
Watsonx Orchestrate wraps a second ring around this. It discovers every agent in the environment so each has an owner, including ones built on Amazon Bedrock, Microsoft Foundry or Google’s platform. It runs them in isolated sandboxes with a kill switch, so an agent that starts rebooking every guest on tonight’s manifest because a broken feed reported a 400-minute delay is stopped as a whole, not call by call. And it evaluates agents before and during production, which is where the hotel learns that last month’s rebooking accuracy was 97% rather than assuming it.
Terraform policy is the same control for infrastructure: A proposed change is checked against the organization’s rules before it is applied, so the SRE agent’s security group change is blocked if it would cut off the payments API, whether or not the agent noticed.
What makes this layer deterministic is that nothing depends on the model being right. The credential, the sandbox and the policy all say no to the same request every time.
Layer 4: What gets recorded
Layers 1 to 3 prevent most wrong actions. Some still get through: The facts were current, the function was the right one, the credential allowed it and the model still chose badly. Layer 4 exists so those cases can be found by machine rather than by waiting for complaints.
Two things make that possible. First, every agent action is logged under the agent’s own identity with the inputs it read, the rule it applied and the function it called. That is a by-product of Layers 1 to 3, and Watsonx Orchestrate keeps it. Second, IBM Concert correlates those agent logs with the operational signals from everything else: application performance, infrastructure changes, security events. On its own, an agent log tells you what the agent did. Next to the infrastructure and security timeline, it tells you why.
Take three guests who complain on Tuesday morning that their shuttle left without them. The agent log shows all three were marked “no action needed” against a shuttle ETA of 20:55. Concert puts that next to the infrastructure record, which shows the traffic feed stopped updating at 20:40, and the security record, which shows Vault rotated that feed’s credential at 20:39 and the feed never picked up the new one. The shuttle actually left at 20:35. Three complaints, one root cause, found in minutes from logs the system wrote, not from a post-mortem with the agent. The SRE fixes the credential handoff, and the agent re-runs the three cases.
This is the layer that turns the 141 from a number into a list, and the list is what lets a human sign off on running the agent again tomorrow.
Scalability impact of the 4 layers
Each of the four layers takes one type of decision away from the model and hands it to a component that returns the same answer every time. A lookup provides the facts, a catalog provides the functions, a rule decides what is permitted and a log records what happened. Lookups, catalogs, rules and logs cost the same at 1,000 agents as at 2 million, and they return the same result at both scales. The model’s own error rate does not improve. What improves is how rarely the model is asked to guess and how quickly the guesses it gets wrong can be found and reversed.
This matters because the difference between a pilot and a production rollout is supervision. During a pilot, a team watches every rebooking the agent makes and manually corrects the wrong ones. Nobody can watch 150,000 agents, so the stack has to catch the 141 wrong rebookings before the guests reach the curb, or at least list them by the next morning. Enterprises will roll out agents for workflows that touch customers, money and production systems once they trust that a wrong answer stays small and shows up in a log. The four layers IBM presented create a credible architecture for that.
It is great to see IBM address the scalability problem at the level of data, functions, permissions and records, rather than at the level of the model. IBM will have to prove that the four layers work together as one system. Infragraph and Streamhouse do not yet update on the same clock; Intelligence Hub is in tech preview until December; and no customer has shown all four layers running end-to-end. The 22% of enterprises that already run agents in mission-critical production built their own version of this stack. The question for the remaining 78% is whether assembling IBM’s version is faster than building their own, and that is something reference customers will have to show over the next year.
Torsten Volk is principal analyst at Omdia covering application modernization, cloud-native applications, DevOps, hybrid cloud and observability.
Omdia is a division of Informa TechTarget. Its analysts have business relationships with technology vendors.
