Getty Images/iStockphoto
Unstructured data is the bottleneck for agentic AI success
Ungoverned, unstructured data blocks enterprises from scaling agentic AI. Contracts, transcripts and internal documents hold the context agents need but can't yet reach.
The pressure to move quickly on AI is causing many enterprises to choose speed over governance. It may accelerate a pilot, but it won't scale AI. Without confidence in the underlying data, agents can't act responsibly or reliably.
In Collibra's recent research with The Harris Poll, nearly 90% of tech decision-makers said they can't fully trust AI-driven insights until the data behind them is verified through formal governance. This is the trust gap in AI, and it's one of the reasons organizations struggle to move AI from pilots into production. And despite significant investment in AI, the consequences are already visible.
Recent McKinsey research found that only 7% of companies have fully scaled AI across their businesses. That number won't change until enterprises unlock one of their most overlooked assets: unstructured data.
Unstructured data makes up between 80% and 90% of enterprise data and holds much of the business context AI agents depend on to reason, make decisions and act. Yet much of it remains unused. If that doesn't change, agents won't just fail at the build and design stage. They'll fail at deployment. Speed without governance isn't speed. It's simply delaying the problem until production.
Governance never covered the data that matters most
Historically, enterprises have built governance around the data that is easiest to manage: the structured rows and columns stored in databases and warehouses. Unstructured data was left behind.
For two decades, it simply wasn't workable. BI tools couldn't read it. Catalogs couldn't index it. Governance frameworks were never built to cover it. That gap didn't matter much when AI was mostly generating text or summarizing what a person fed it directly. It matters much more now that agents are expected to find information across systems, connect it and act on it. The quality of those actions depends on more than the model itself. It depends on whether the enterprise has made its knowledge discoverable, governed and available in the right context.
The proprietary data that makes an enterprise unique, including contracts, transcripts and internal documents, has gone ungoverned for years. This has meant that agents can't use the very thing that would make them even more valuable. Without it, an agent has no real advantage. It's just another general-knowledge LLM with extra steps.
Structured data answers the easy questions
Consider a customer support agent investigating a complaint about a defective product. The order record is the easy part. It's structured, stored in a database and any AI system can retrieve it in seconds.
What actually determines the outcome is harder to find. The documentation is buried in a shared drive. The support tickets were filed months ago. The email thread where a field engineer diagnosed this same issue last year. Individually, each source tells only part of the story. Together, they provide the context needed to determine whether the issue is isolated, part of a known pattern, or one the organization has already solved.
The answer isn't waiting for better technology. The technology already exists. Today, AI can generate metadata, identify sensitive information, classify content and assemble the specific information an agent needs to do its job.
This matters because cost is just as important as capability. An LLM can process enormous amounts of unstructured data, but the token cost is significant and the results are often inconsistent. Feeding an agent everything is not the same as feeding it the right information. When an agent gets the information it actually needs, rather than the entire stack, it delivers better answers at a fraction of the cost. That’s the difference between an agent workload that scales and one that quietly becomes too expensive to sustain.
Unstructured data can't wait for a perfect roadmap
Enterprise leaders don't have the luxury of choosing between speed and governance. Manual classification was never going to keep pace with the volume of unstructured content organizations generate every day, let alone across the year. At the same time, waiting for a perfectly governed data set before deploying AI isn't realistic either. The organizations that succeed will be the ones that treat unstructured data as core AI infrastructure, not something to address later.
The path forward isn't the same for every organization. Some will build these capabilities internally. Others will buy them or partner with them. What matters is treating unstructured data with the same discipline that's long been applied to structured data. Contracts, call transcripts and internal documents now matter just as much as databases because AI agents depend on both. And token economics shouldn't be an afterthought. The cost of running agents on ungoverned data only compounds as more agents are deployed.
None of this requires a perfect data set or a finished roadmap. It requires a decision to stop treating unstructured data as something that can wait.
The organizations that make that decision won't build agents that work in a demo. They'll build agents that understand their business because they can access the knowledge, context and expertise that already exists across the enterprise.
Felix Van de Maele is co-founder and CEO of Collibra and was named EY Technology Entrepreneur of the Year in 2019.
