The work of data teams covers a lot of different areas, from data ingestion through analysis, but the newest technologies and how they're used revolve around a common theme: organizations must rethink data architecture around the demands of agentic AI workflows.
Decentralized data access, ontology layers, shift-left governance, real-time data delivery and cost control are all responses to the underlying challenge of data simultaneously requiring context, speed and cost discipline. And none of these can be addressed in isolation.
The most significant element of change is the shift toward proactive data management as both technological and business strategy. Instead of reactive processes meant to clean things up after they happen, data professionals are moving upstream to get ahead of the increasingly complex demands being made of an organization's data, which is its most fundamental and dynamic asset. Read on for five key trends in 2027.
Agents further decentralize data
Rather than moving data through complex and costly extract, transform and load (ETL) pipelines into centralized data warehouses, lakes and lakehouses, organizations are increasingly keeping their proprietary data in myriad systems, including SaaS applications such as Salesforce and SAP where it’s created. Initially, humans were the end users empowered to query information spread throughout their organization’s data estate.
Now, that trend is being extended to AI agents.
Agents are being asked to execute business processes rather than simply respond to a user’s query. Such workflows generally require data from more than one source. Theoretically, enterprises could build ETL pipelines that bring together data from each source for agents, but that’s time-consuming, complex and expensive. Plus, it exposes data to potential security breaches.
The answer is to enable agents to call on data where it lives.
“I expect access to data where it lives to become table stakes across enterprise data and AI platforms — in fact, you could say it already is table stakes,” David Menninger, an analyst at ISG, told TechTarget. “Agents increase the urgency because business workflows often span multiple systems, and requiring a separate integration project for each workflow slows deployment.”
However, Menninger noted that there is more to building agents that can be trusted in production than connecting them with the right data, irrespective of where it lives. Context must also be part of the equation. And while enterprises are figuring out how to connect agents with their data, connecting agents with relevant context is an ongoing challenge.
“There is still a need to distinguish between making data accessible and making it usable,” Menninger said. “Agents need consistent definitions and context. Sharing data is not sufficient. Context needs to be shared as well. That’s hard to do right now because enterprises haven’t fully tackled the issue of context yet.”
The ontology layer becomes more important
Semantic layers have long been incorporated into BI platforms, enabling users to standardize how business metrics are calculated and what different terms related to their data mean. Semantic layers alone, however, have not enabled enterprises to connect agents with context that’s spread across disparate systems — such as data warehouses, lakes and lakehouses, databases and SaaS applications — to successfully build agents and other AI tools.
Enter the ontology layer, which is not new but is taking on added importance.
Ontologies are formal models that define the core concepts and metrics of an organization, such as its products and customers; define the rules that govern the relationships between core concepts; and directly connect those formal models to underlying data. In effect, semantic models define consistent data terms across the business, and ontologies go further by modeling how businesses truly operate.
Agents need to understand not just how revenue is calculated but deeper details such as which customers use which products, what other products those customers may be entitled to use, and how the relationships between customers and products affect business decisions. As a result, ontologies are becoming more popular.
And as enterprises look to put more agents into production, providing a single layer within storage platforms that all agents can call on through Model Context Protocol (MCP) servers to discover the context they need will continue to gain popularity, according to Michael Ni, an analyst at Constellation Research.
“Expect ontologies to gain traction in 2027 as enterprises move more AI agents into production,” he said. “Agents working across multiple systems need a shared understanding of business entities, relationships and rules. … My broader prediction is that enterprise context will become a critical part of AI infrastructure.”
Vendors such as Databricks, which made Genie Ontology generally available on Nov. 11, and Snowflake, which introduced Business Ontology – soon to be in public preview – on Nov. 3, are adding capabilities that enable users to create ontology layers. In addition, Microsoft and Palantir are among those now prioritizing business ontologies, while specialists such as Graphwise and Timbr were among the earliest providers of ontology layers.
“Palantir’s commercial success has made the concept [of an ontology layer] attractive,” Donald Farmer, founder and principal of TreeHive Strategy, told TechTarget. “Also, language models have reduced the cost — the old objection to ontologies was the manual modeling — and newer products [such as Genie Ontology] are more self-developed and self-sustaining.”
Data governance continues to shift left
The traditional data governance process is reactive: data issues are identified and governance policies are applied after systems have been deployed. But that approach doesn’t meet the AI moment, according to governance leaders and data management consultants.
Enter shift-left data governance. It incorporates governance into data architecture design and application development workflows from the start, providing more proactive data quality checks, lineage documentation and policy enforcement as AI systems are created.
“From a data flow perspective, you have an opportunity to embed governance all along the way,” Brandy O’Shields, senior manager of enterprise data governance at PayPal, said during Dataversity’s Data Architecture Online 2026 conference in July.
Anne Marie Smith, principal consultant at Alabama Yankee Systems, disdains the shift-left term. But she said she and other consultants have recommended for more than a decade that governance should be built in upfront rather than addressed after implementation. It’s now happening more frequently as organizations recognize the need and the potential business benefits, Smith said, although she added that most organizations are still focused on traditional governance.
William McKnight, president of McKnight Consulting Group, similarly said shift-left governance isn’t the default approach yet. But he added that it’s rapidly approaching mainstream adoption among leading enterprises — particularly ones with successful AI deployments.
To make a shift-left approach work, both Smith and McKnight said data governance leaders need to position the governance process as an enabler of better data use and business innovation, not just an approval and control layer. Other key elements include data contracts that define schema and metadata requirements, automated data profiling and classification, policy-as-code implementations and continuous data observability. Managed effectively, shift-left data governance enables organizations to “scale AI safely without sacrificing development velocity,” McKnight said.
AI agents’ demand for real-time data creates new governance challenges
The ability of AI agents to analyze and act on data at machine speed puts increased pressure on data teams to provide them with trustworthy real-time data, said Stephen Catanzano, a senior analyst at Informa TechTarget’s Omdia division.
“An agent acting on yesterday’s data makes yesterday’s decisions,” he said. “In 2027, real-time data will shift from a specialized capability to a baseline expectation for any organization putting agents into production.”
Omdia survey results published in October 2026 reinforce that prediction: 50% of the 385 IT and data professionals who responded said real-time streaming is now the most common way their organization delivers data to AI tools, ahead of scheduled batch processes at 38%.
Data management vendors are rushing to roll out new technologies for delivering real-time data to AI agents. That includes Databricks and Snowflake, as well as various database and streaming data platform vendors.
The rub is that real-time data used by agents must also be governed at machine speed — and traditional data governance isn’t built for that. In the Omdia survey, only 7% of respondents said their governance programs are designed to support real-time, automated processes. Nearly six in 10 said adapting governance for AI agents is a very or extremely significant challenge in their organization.
Catanzano said data quality checks must be built into streams to prevent bad data from reaching agents. Comprehensive lineage and audit trails are also required, he added, and access controls “must be enforced at the moment an agent requests data” rather than through static roles and permissions.
Farmer said the governance challenges are exacerbated by the fact that agentic AI use cases tend to be more unpredictable than existing real-time applications with human users. Modern data platforms simplify the process of building real-time data pipelines for AI, he said, “but there is more to govern.”
Data costs become a priority as AI deployments increase
Credit or blame AI: While approximately a third of enterprises have reported deploying AI agents beyond the pilot stage — Gartner’s 2027 CIO and Technology Executive Survey reported that 34% of organizations had deployed AI agents to date — the data costs are already piling up.
Per data management vendor Fivetran’s global survey of 500 senior data and technology leaders for its 2026 enterprise data infrastructure benchmark report, published in March, annual data budgets averaged more than $29 million per organization.
Without strict unit economics tracking — cost per query or agent run — and more proactive data management strategies to reduce the cost of remediating bad data, runaway data processing, storage, token and compute bills could impact IT budgets and data professionals’ jobs.
"AI is moving from pilots to production, and the economics change completely when it does,” said Sanjeev Mohan, founder and principal of analyst firm SanjMo. “In a pilot, nobody watches the bill. In production, agents run continuously. They query data in loops, retry, and call many tools to answer one question. With consumption pricing, costs now scale with agent activity."
Effectively managing those costs is becoming more critical, Mohan added. "The blank-check, 'tokenmaxxing' era is ending," he said. "CFOs are demanding to see value. Cost is no longer an operations problem to fix after the fact. It’s an architecture decision that has to be made upfront."
In 2027, watch for cost control to move upstream in the data lifecycle. As highlighted by the FinOps Foundation, organizations are shifting away from bill-cutting toward pre-deployment cost forecasting, query pruning and semantic context layers. Gartner notes prioritizing semantics could reduce AI data costs by up to 60%.
And if these costs aren’t addressed proactively, Mohan warned, the fallout can affect the whole enterprise.
"Finance will impose blanket budget cuts and freezes, and force vendor consolidation from the top down," he said. "Good AI projects will get cut along with bad ones, because no one can tell them apart. The data team gets labeled a cost center instead of a value driver. And business units will route around central data teams to buy their own AI tools, creating the very sprawl that drove costs up in the first place."
