Databricks intros new Genie, data management tools to aid AI

Progress toward problem solving

Despite enterprises investing heavily in developing agents and other AI applications, vendors creating environments designed to enable customers to build agents using an organization's proprietary data, and advancements in AI model capabilities, most AI initiatives fail.

Databricks [has] its finger on the pulse of what customers need to effectively manage their data to drive the outcomes they desire. The announcements focus on meeting the needs of clients. … Each addresses the challenges customers are struggling with to operationalize AI.

The results of surveys examining the success rate of AI initiatives vary.

A 2025 study by MIT found that 95% of organizations were getting no return on their investments in AI development, while a 2026 study by Deloitte found that 25% of companies are now putting agents into production. Though the findings are different, they each show that most AI initiatives still don't move past the pilot stage.

One of the main contributors to the high failure rate is that organizations struggle to connect agents with the context -- the situational awareness that comes from data and business logic -- they need to properly perform.

Without appropriate context, agents will fill in gaps with an inference based on what information they do possess, which can lead to incorrect and misleading outputs that could prove costly for organizations.

The new capabilities unveiled by Databricks are designed to help customers organize, discover and deliver relevant context to agents precisely when they need it, according to Ali Ghodsi, co-founder and CEO of Databricks.

"Getting the perfect enterprise context is elusive, and this is the problem we're focused on at Databricks," he said during Data + AI Summit's keynote address. "Context … means getting all the data you have in your organization and connecting it to the AI."

To do so, data needs to be AI-ready, governed and secure, cost-effective to operationalize, and the workflow needs to be executed in a flexible manner so organizations can make choices as technologies evolve, so they don't get locked into an inflexible AI stack, Ghodsi continued.

"We're making steps on how to solve these four," he said. "I'm not going to say we've completely solved context, but at least we're making some big leaps. We want you to have choice, we want to provide security and cost control and get the context to the AI."

Of all the capabilities Databricks unveiled during its conference, Genie Ontology most directly addresses connecting agents with appropriate context.

Genie Ontology is an automated context layer that finds and delivers relevant information from tables, dashboards, pipelines and applications. Once Genie Ontology extracts relevant information, it organizes it into a knowledge graph that enables agents to understand what information means within the context of an enterprise.

Genie Ontology then provides situational awareness to other new Databricks features such as Genie One, an AI-powered coworker that includes connectors to provide insights from across an organization's data estate, and Genie Agents, which are domain-specific agents that use the same tools as Genie One, but autonomously execute business processes rather than work in conjunction with humans.

Michael Ni, an analyst at Constellation Research, noted that Genie Ontology -- in conjunction with Lake Transactional Analytic Processing (LTAP), which is the architecture in Lakebase that unifies data types -- addresses one of the main problems enterprises face during AI development.

"Those two capabilities address the two biggest barriers to enterprise AI, which are making it easier to give agents trusted business context that learns with every transaction, and real-time operational data that is ready to be acted upon," he told TechTarget. "Everything else … builds on those foundations."

Leone likewise highlighted the value of Genie Ontology, if it performs in production as Databricks intends.

"The hard part of letting everyone talk to their data has always been getting the system to understand what a company actually means terms like 'active customer' or 'margin,'" he said. "If they've genuinely cracked that, a business user can act on the answer without stopping to verify it, and most conversational tools still can't promise that."

Keeping pace with competitors

In addition to the new Genie features, the following new Databricks capabilities help provide agents with proper context:

• LTAP, a processing architecture within Databricks' PostegreSQL database, readies data for AI by unifying transactional, analytical, streaming and operational data in a single location without having to duplicate or move data.
• Lakehouse//RT, an evolution of Databricks' data lakehouse powered by the vendor's new Reyden compute engine that enables customers to run real-time workloads, also prepares data for AI.
• Cost control capabilities in Unity AI Gateway, a governance control plane for AI assets within the Databricks Unity Catalog, that improve spending visibility, intelligently route workloads and allow organizations to set hard spending caps.
• Integrations with AI security, identity governance, data protection and threat detection platforms that add to the openness of the Databricks ecosystem.
• Extension of the Unity Catalog to govern assets beyond models, including agents, agent skills and Model Context Protocol servers.
• AI monitoring and investigative capabilities to provide visibility into agentic workflows.

Beyond capabilities aimed at connecting agents with context, Databricks introduced CustomerLake, an agent-powered customer data platform that marks Databricks' first foray into the marketing industry.

Leone highlighted the importance of Unity AI Gateway given that cost concerns contribute to the high failure rate of AI initiatives. However, he noted that Genie Ontology and the other new features Databricks unveiled are largely in step with those that other vendors are offering rather than differentiated or inventive.

"Most of it is in line with where the whole market is heading," Leone said. "Everyone right now wants to be the governance, context, and cost layer over all of a customer's tools and distributed data, so that ambition isn't what sets Databricks apart. Where they're genuinely different is how open they stay underneath it all."

Ni similarly noted that where Databricks distinguishes itself is with its architectural designs rather than individual features.

"Most major platform vendors now offer agents, copilots, governance and AI development tools," he said. "What's differentiated is the architecture. Databricks is tackling some foundational challenges and redesigning the data platform itself for an agentic world. ... Whether that architectural vision becomes a competitive advantage now depends on execution."

Beyond the Summit

After addressing context for agents with Genie Ontology and its latest array of new features, Databricks still has an opportunity to make its many data management and AI development capabilities easier for customers to operationalize in conjunction with one another, according to Ni.

"Over the last several years, Databricks has simplified the data and AI stack by unifying [capabilities] on a common foundation," he said. "As organizations move beyond platform adoption to enterprise transformation, they need proven implementation blueprints, governance models, training, and change management that reduce deployment risk and accelerate time to value."

Catanzano likewise suggested that Databricks address simplification, though rather than focus on the ease-of-use of its own platform, he advised the vendor to address the data isolation problems many enterprises struggle to overcome.

"There is still a major challenge of data silos where the data you need for an AI solution is spread out everywhere," he said. "These and other announcements move toward breaking that down with the Lakhouse being the central repository. Executing on this is not easy and likely the focus for a while."

Eric Avidon is a senior news writer for Informa TechTarget and a journalist with more than three decades of experience. He covers analytics and data management.