Enterprises building a modern data architecture have a host of frameworks and data platform components to consider. Each layer adds complexity, and that complexity gets expensive fast when something breaks.
The components of today's infrastructure include data meshes, data fabrics, data sources and ingestion tools, data warehouses, data lakehouses, analytics capabilities and governance layers – each serving its own purpose and bringing its own dependencies and potential points of failure. The management of these complex environments is a challenge. This difficulty is most apparent when problems occur in the form of broken data pipelines, data leaks or governance rule violations. When data and its processing are scattered across platforms, the signals required for root-cause analysis can prove difficult to isolate.
"The proliferation of distributed environments makes the job almost infinitely more complex," said David Marco, president and executive advisor at EWSolutions, which focuses on data and AI governance.
AI's evolution introduces additional data complications. With GenAI, organizations can limit exposure by restricting prompts or curbing user interactions with a model. But agentic AI has brought new and unsettled territory for enterprises, with a continually evolving set of governance and traceability challenges.
Lauren Kornutick, a senior director analyst in Gartner's Analytics and AI team, has seen that shift among the market researcher's clients.
"When they were just experimenting with generative AI, they didn't have some of the data issues they're bumping into now," she said. "What we're seeing happen with agent deployments is the data challenges are becoming bigger."
Against this backdrop, a key question has emerged: What traceability capabilities should data leaders require before they approve a new data architecture or platform layer? Enterprises can answer that question in various ways. Their approaches depend on the problems they're trying to solve and the data and AI management roles they plan to automate.
Data managers should ask several follow-up questions when deciding which traceability option to deploy.
"At a high level, before approving a new data architecture or platform layer, I would want to know whether the organization can reconstruct the life of a critical data element after the fact -- not simply determine where the data resides today," Marco said.
Know your source before you trust the data
Reconstructing the data lifecycle breaks down into several more specific questions, starting with where the data originates. The architecture should identify the authoritative source or system of record, the data owner, provenance, the creation or ingestion point, classification and data quality status, Marco said.
Provenance must be captured at ingestion, said Amit Chandak, chief analytics officer at Kanerika, an AI, data engineering and migration consulting firm.
"Reconstruct it afterwards and you are guessing from old Slack threads and whoever happens to remember," he said.
Provenance should inform architecture and platform decisions.
"Before you approve a layer, check that a dataset cannot enter it without a named owner, a source system and a sensitivity classification attached at the first hop," Chandak said. "Make those required fields that fail the pipeline when they are missing."
Insist on column-level rather than table-level lineage. Column-level lineage supports root-cause analysis.
"At 2 a.m., you need to know which field produced the wrong number," Chandak said, adding that banks subject to Basel Committee on Banking Supervision 239, a data and risk management framework, already must trace critical data elements at that granularity.
What happened to data in its journey?
Another important consideration is how data was moved or transformed, Marco said. Organizations should look for end-to-end, automated lineage across pipelines, integrations, APIs, ETL and ELT processes, replication and transformations. That scope should include the business or technical rules that are applied and, where appropriate, the version of those rules, he added.
Lineage inside an individual platform is usually acceptable because the vendor can read its own metadata. The breakdown occurs at the seam between tools.
To that end, Chandak said, organizations evaluating a new platform should ask whether it can emit and consume events supported by OpenLineage, an open-source framework and standard used to gather and check data lineage across data environments. Or does the platform only see what it runs? If that's the case, then every customer script, notebook and reverse ETL hop becomes a gap in visibility, he said.
Access control difficulties increase with AI agents
An organization should be able to automatically determine who or what accessed the data, which policy and policy version applied, what decision was made and whether an exception, override or approval occurred.
Policy versioning can be a sticking point since buyers often forget to ask vendors about it.
"A useful test during an evaluation is to ask the vendor to pull up a denied access request from 60 days ago with the policy version attached," Chandak said.
In the case of agentic AI, top issues include identity and access, Kornutick said. With identity, how do organizations ensure that one user's agent doesn't access information that only another user's agent is allowed to obtain? Or if two employees are working with the same agent, how does the organization ensure there's no leakage between the respective users' permissions?
As for access control, enterprises are usually in better shape with structured data versus unstructured data, she said. Unstructured data isn't classified, so internal information leakage can occur.
"Guardrails run on metadata that classifies unstructured data," she said, adding that organizations find it hard to get their arms around classification.
Gartner clients have begun to explore using software to automate metadata tagging across their unstructured data, Kornutick said.
What vendor demos leave out
When an incident occurs, identifying the affected downstream data products, reports or business processes becomes critical. But the platform's ability to isolate impacted resources might not get the attention it deserves during the selection process.
"Blast radius is the capability that gets the least demo time," Chandak said, adding that's the capability data leaders rely on most once an incident occurs.
"Most lineage tools stop at the edge of the data platform, and your exposure keeps going into BI reports, embedded dashboards, downstream applications and the business processes sitting on top of them," he said.
Organizations should structure their vendor evaluations by inviting reps to list every asset and process downstream of their product, including elements outside the vendor's offering, Chandak said. Evaluators should watch how much of the vendor's answer relies on manual processes, he added.
Marco also stressed the importance of quickly understanding the blast radius. Strong traceability gives impact analysis across data products, reports, analytical models, AI systems or agents, applications and critical business processes. That requires a historical record: metadata, policies, transformations, ownership and controls all need enough history and versioning to recreate what actually happened at the time.
Picking the right tools for traceability
Enterprises rolling out AI must build a traceability layer within their data architectures. A wide range of products support traceability functions, among them metadata management platforms, data security posture management, AI gateways and, to some extent, privacy-enhancing technologies that can assist with filtering.
Kornutick said AI governance platforms help organizations determine which agents to deploy, set guardrails and detect shadow AI. Observability tools track how agents behave, while AI application security tools provide in-line runtime inspection and enforcement. Regulated enterprises may use cryptographic tracing, which can verify how an agent outcome was produced.
Most enterprises will likely use multiple tools for traceability.
"I don't think that you are going to be able to get one tool to trace every aspect of the AI lifecycle equally well," Kornutick said, adding that tool selection will vary by organizational role.
AI governance teams will assess discovery and registry, policy management and run-time enforcement. Data teams will also need AI discovery and registry capabilities, but they will be more interested in data traceability and data quality.
There is no single right way for organizations to handle traceability, Kornutick said.
"Everyone feels like they're behind, but it's not because of them necessarily. It's because the AI is moving faster than we can figure out how to build the governance and security around it," she said.
John Moore is a freelance writer who has covered business and technology topics for 40 years. He focuses on enterprise IT strategy, AI adoption, data management and partner ecosystems.