Data management vendors race to connect AI with context

An array of announcements

Models, as it turned out, weren't the only problem.

For AI tools to be trusted, reasoning capabilities, whether provided by large language models (LLMs) such as ChatGPT and Claude or smaller, purpose-built models, need to be strong for an AI tool to respond accurately to queries. However, even as LLM reasoning capabilities close in on 100% accuracy for basic queries, most AI initiatives still stall.

Without it, they're guessing. Guesswork doesn't work for enterprise businesses.

Context remains the missing piece.

"Without context, an agent is at risk of taking the wrong action, not necessarily because it malfunctioned but because it didn't have the information it needed to make the right decision," David Menninger, an analyst at ISG Software Research, told TechTarget.

To improve data discovery and retrieval, data management vendors have centered product development efforts on enabling customers to find and feed agents the context they need.

A week after Databricks' early January launch of Instructed Retriever, which improved on traditional retrieval-augmented generation (RAG) by adding parameters such as user instructions to searches to enhance retrieval accuracy, MongoDB unveiled new vector embedding and reranking models to improve the relevance of data discovered using vector search.

Among many others, vendors such as Alteryx, AWS, Google Cloud, GoodData, Informatica, Snowflake, Tableau, Teradata and ThoughtSpot have followed suit.

Not all have taken the same approach.

Databricks aims to simplify and improve data retrieval systems. Like MongoDB, Teradata is among the vendors attempting to improve vector search to discover and operationalize context so Instructed Retriever, RAG and other retrieval pipelines can pull it in. And many are adding or augmenting semantic modeling capabilities to enable users to build context layers for AI.

In June alone, Databricks, AWS, Microsoft and Snowflake revealed new tools that help customers create context layers for agents.

The organizations that proactively went through the painstaking process of putting their data estates in order and building infrastructures that can handle the speed and scale of AI workloads are putting at least some agents into production.

For example, Deloitte, which in January reported that one-quarter of organizations were able to move 40% of their AI projects into production, predicted that around half would reach that 40% mark by midyear. Similarly, Databricks' 2026 State of AI Agents report, published in January, showed a 327% increase in the number of organizations transitioning from single chatbots to multi-agent systems over the previous four months.

"When agents are given clear, sufficient context, they can complete tasks in fewer steps and reach the right data faster, rather than spending cycles exploring dead ends," Bendersky said.

Successes, however, still represent a minority.

Semantic strength

With many enterprises aiming to build broad, multi-agent systems, improving vector search and RAG are key to connecting agents with context. However, within this year's dominant data management trend, semantic modeling has emerged as the most common approach to improving context retrieval.

Without semantic layers, LLM accuracy drops sharply when asked to derive outcomes from data distributed across multiple systems. But when semantic layers augment LLM queries, they recover their accuracy.

Cindi Howson

"Semantic layers are the magic," Cindi Howson, chief data and AI strategist at ThoughtSpot, told TechTarget. "[Accuracy] depends on how much context you give agents and how robust the semantic layer is, but semantic layers are the difference. The variability will be the complexity of the data model, as well as how robust that semantic and context layer is."

Like vector embeddings, semantic modeling makes data discoverable by assigning definitions and characteristics that ensure consistency across an organization. But similar to vector search and storage, semantic modeling was, until recently, a niche capability used to augment analytics systems rather than a key component.

Now, semantic layers are becoming so critical to agent performance that a group of data management and analytics vendors has banded together to simplify semantic modeling by creating an open industry standard. Without one, vendors provide proprietary semantic layers that can't interact with one another, adding complexity rather than reducing it as developers and engineers piece together AI pipelines that connect data stored in myriad systems.

"There's the whole notion of what the data means, and we've never been very good at that," Menninger said, noting that there is no single, universal semantic language. "We had the same issue in the BI world as we do in the AI world. … We need a way to describe what the data means."

One enterprise deploying a semantic layer to help organize its data and make it available for analytics and AI applications is supply chain management specialist Blue Yonder, which uses semantic modeling provider AtScale's platform.

With At Scale's capabilities, Blue Yonder in July 2024 rearchitected its previously disparate data using semantic modeling, then connected its unified semantic layer to AI tools with AtScale's Model Context Protocol server to enable agents to call on contextually relevant data.

"It allows us to get that clear alignment between the data and the business, and really define everything in one single place," Jeremy Arendt, senior director of analytics engineering at Blue Yonder, said at a session during AtScale's virtual Semantic Layer Summit in May. "The semantic layer became an integral part of making sure we have trust and faith in that data."

Now, queries that previously took days to answer happen in near real time, he continued.

"Our semantic layer is for everyone in the company who needs data," Arendt said. "We're focusing on enabling self-service [insight generation] at every layer of the business."

Other tools of the trade

Though semantic layers have emerged as a key aspect of context retrieval for AI, they are just one part of a larger whole.

Semantic layers map enterprise schemas to make relevant data ready for agents. Also critical are instruction-aware retrieval layers and zero-trust governance and security capabilities that treat every user, device or application attempting to access an organization's data and business logic, according to Farmer.

Data governance establishes policies and standards that determine how an enterprise's data is managed and accessed by users, including agents. Security capabilities, meanwhile, protect organizations from potential misuse by users, including agents, as well as attacks from outside threat actors.

Donald Farmer

Recently, with agents introducing new data security concerns by increasing the scale and speed with which data is accessed and operationalized as well as enabling attackers to increase the speed and scale of threats, Oracle emphasized the need for enhanced security. However, instead of applying security measures to the operating systems historically protected by such tools, Oracle determined that security now needs to be applied at the data layer to better protect against increased threats. 

"Advanced retrievers treat business constraints … as strict rules rather than just suggestions for the LLM, [and] zero-trust agent security frameworks ensure that agents inherently use and cannot exceed the security clearance of human users who initialized them," Farmer said.

Meanwhile, all the components that comprise a context retrieval system for AI -- those that foster discovery, retrieval and action -- need to be in a single environment, according to Bendersky.

If they aren't delivered as a unified capability, connecting agents with context becomes a complex engineering exercise that has to be manually recreated for each use case.

"Context delivery needs to be a platform capability, not an engineering project," Bendersky said. "This means access controls, lineage and auditability need to be built in… because how your agent got its context matters as much as what it said."

Eric Avidon is a senior news writer for Informa TechTarget and a journalist with more than three decades of experience. He covers analytics and data management.