AI data fabric emerges as a governance layer for agents

Why AI data fabrics are drawing attention

Agentic AI has made architectural decisions more urgent. CIOs and other technology leaders face a complex management challenge as more autonomous agents move into production.

The modern AI data fabric extends beyond data integration by adding semantic context and governance layers needed for these autonomous agents. Potential benefits include support for data products, flexibility to adapt to changing technologies and improved observability.

"You need some kind of an enterprise data fabric to be the path between the agents that people are trying to build and all of that enterprise data that is potentially siloed," said Kevin Keuning, a senior vice president at NWN, a technology services provider.

This AI-oriented fabric model expands the role of earlier data fabric designs.

"Traditional data architectures focus on storing, moving and organizing data," Safari said. "An AI data fabric adds the context, intelligence and governance that AI systems and agents need to operate effectively across the enterprise."

How data fabrics help agents do their jobs

AI data fabrics can help facilitate agentic AI rollouts through a centralized governance layer that spans agents and the data they use.

"Without that kind of a layer, you would need to look at each individual silo of data and then determine the right path for agents taking advantage of that data," Keuning said.

AI data fabrics also serve up business context, which is critical for agents making decisions or recommendations. To do that, the architecture can apply a common business vocabulary on top of an enterprise's domains -- a bank's mortgage or credit card businesses, for instance -- so an AI agent can understand the data within those domains, said Pramod Sadalage, Distinguished Engineer at Thoughtworks, a technology consulting company.

This vocabulary includes domain concepts, definitions, relationships, policies and business meaning, supported by a semantic layer, he said. This approach gives people and AI agents a more consistent way to interpret enterprise data, he added.

But Sadalage pointed to a crucial difference between the two.

"When you put data in front of humans, they have a bunch of context in their heads, being in the business domain for some time, and they understand what it means," he said. "But the AI agent will struggle to understand what the data means if you don't have the right context around it."

An AI data fabric also plays a role in managing identities and access rights as the agent population grows.

"In most cases, the AI data fabric layer can be pretty helpful in making that possible," Keuning said. "That's the layer where we can define that governance around the identity of the human end user, or the agentic end user or a mix of both. That has to be part of the whole architecture."

Sadalage also cited the issue of access rights. An AI agent granted authority beyond a user's privileges could expose data the user is not authorized to see. For that reason, the user's authentication and privileges should be delegated to the agent, he noted.

Delegation happens at the agent layer, where the agent acts on behalf of the user. The fabric responds to the agent as though the user were requesting the data, Sadalage said.

Data fabrics generally play well with other architectures.

Keuning said AI data fabrics and data meshes, for instance, are complementary. Data mesh distributes data ownership and governance by domain. Data fabrics, meanwhile, can provide a unified governance and access layer across those resources to support AI deployments, he noted.

"The data mesh handles local governance within each domain, and the fabric handles global governance across all domains," he said.

Some organizations are already combining data fabric and data mesh principles within a single architecture. For example, Stibo Systems built an internal enterprise intelligence platform on Microsoft Fabric that combines centralized governance and shared semantic models with data mesh principles, such as domain-level ownership and reusable data products.

Thomas Møgelmose, CFO at Stibo Systems, described the model as "a centralized, governed platform combined with decentralized ownership and data product thinking."

Building flexibility for evolving AI architectures

On the storage side, primary repositories within a data fabric can include data lakes and data lakehouses, Keuning said. Other data might enter an organization through APIs, he added.

An AI data fabric serves as an abstraction layer that provides adaptability.

"The fabric exposes a stable governance interface over the data while the underlying systems can change," Keuning said.

In the case of agentic AI, an organization might need to optimize data for vector search or knowledge graphs, Keuning said. Those techniques help AI agents retrieve relevant information with semantics and support reasoning.

That flexibility matters as organizations test different ways to connect agents with enterprise data. Keuning cited agentic retrieval-augmented generation, keyword search and other types of agentic search as examples.

That evolution will continue, as will the need to adjust.

"There will be more approaches," Keuning said. "Having that optionality in the architecture, so that you can change the way data stores are either optimized or presented to the agents, is going to be important."

Obstacles to AI data fabric adoption

Data leaders must consider implementation challenges that could hinder a fabric strategy and its expected benefits.

Sadalage pointed to integration complexity as a potential deployment issue for AI data fabrics.

"The biggest challenge I see is there is no one tool that says, 'Hey, this is the AI data fabric,'" he said.

As a result, organizations might need to pull together an enterprise data catalog, data integration tools, Model Context Protocol servers, a semantic layer and metadata management platforms. Businesses accustomed to creating data products will likely have some of those components. Even so, they could still face obstacles, such as making their data AI-ready, Sadalage said.

Safari identified the semantic layer as the "real differentiator" of an AI data fabric. That layer consists of knowledge graphs, ontologies and business context that help AI systems understand how data relates across the organization, she said. But creating a shared business context through metadata and semantic models can prove a big hurdle, she added.

Overall, the technology challenge is manageable, Safari noted. Structural considerations can loom large, however.

"The bigger challenge is creating trust, governance, and organizational alignment at enterprise scale," she said. "Most organizations don't struggle because they lack data. They struggle because data is fragmented across systems, functions, and ownership models."

In that context, success depends on integrating legacy and modern platforms, she added.

A growing need for observability

Data managers should plan to increase their focus on monitoring and tracing AI-driven workflows.

"With AI agents in the mix now, people are asking for a lot more observability," Sadalage said. "If there is some decision being made by the agent, I need to know what question was asked, what data was it looking at, when was it looking at it and what decision did it make."

Those questions become important when auditors ask about the AI decision trail.

Keuning cited observability as a fabric deployment challenge that can become a benefit if properly implemented. He said enterprises face a transition similar to that of cloud computing.

"When we built a cloud-native architecture, all of our observability tools were impacted," Keuning said. "To some extent, that is happening again with this AI-native architecture."

He said some tools are breaking or weren't designed to work with newer fabric layers. Agent loops, for example, introduce components that existing observability tools were not built to monitor, he noted.

Overcoming the observability obstacle could give organizations an edge.

"If we have an issue with slow response times, data leakage or governance violations, we need to be able to trace that all the way through the layers of this architecture to its root cause," Keuning said.

John Moore is a freelance writer who has covered business and technology topics for 40 years. He focuses on enterprise IT strategy, AI adoption, data management and partner ecosystems.