Streaming specialist Redpanda adds governance to AI suite
New Agentic Data Plane features enable users to create a governance layer for agents and could help the vendor differentiate itself from its closest competitors.
Streaming data specialist Redpanda on Wednesday launched new features in its Agentic Data Plane aimed at enabling customers to create a unified governance layer for managing connections between agents and data sources.
However, capabilities that govern those connections were not yet ready.
Now, the vendor is adding features such as AI Gateway to provide users a centralized governance pane, AI observability via the OpenTelemetry Protocol (OTLP) to inspect and monitor agent behavior and new security controls.
Given that agents and multi-agent systems require proper governance frameworks to ensure that they act in accordance with an enterprise's policies and meet regulatory requirements, the new ADP features are significant for Redpanda customers, according to William McKnight, president of McKnight Consulting.
"The ADP has the potential to transform Redpanda from a simple streaming engine into a centralized governance layer for enterprise AI," he said. "The update addresses critical barriers by providing unified security and operational control over AI costs and token budgets. This update enables 'glass box' visibility and framework flexibility, allowing users to move … from risky experimentation to secure production."
Based in San Francisco, Redpanda provides a streaming data platform that enables users to capture and process data to fuel real-time analysis. Like many data management providers, the vendor has responded to increasing interest in AI development and added tools that let customers connect data to agents and other AI applications in addition to traditional data products such as dashboards and reports.
Keeping control
Agents need governance.
The ADP has the potential to transform Redpanda from a simple streaming engine into a centralized governance layer for enterprise AI.
William McKnightPresident, McKnight Consulting
Unlike chatbots and other AI applications, agents are capable of autonomous behavior. Rather than requiring user prompts before taking action, they can be trained to work independently.
For example, they can constantly analyze an enterprise's data estate to surface insights that a human analyst might never have discovered. They can take on repetitive, menial work so that employees can be more efficient and spend time doing more meaningful work. And they can work together to optimize complex processes such as managing supply chains.
Because they can make entire organizations better informed and more efficient, agents have been the dominant trend in AI development over the past two years. But because just one agent taking the wrong action can cause significant harm, strict policies and procedures must be in place to ensure that agents can be trusted when put into production.
With most AI initiatives never making it past the pilot stage, a mix of customer feedback and Redpanda's own experiences developing agents provided the impetus for adding new ADP features designed to engender trust that agents will act as intended once deployed, according to Tyler Akidau, the streaming data vendor's chief technology officer.
"Our roadmap has been developed in response to direct customer feedback, our own experiences developing and deploying agents internally and our vision for what is needed to unlock agentic AI in the enterprise," he said.
AI Gateway acts as a unified access layer for connecting agents with AI models and MCP servers by centralizing routing data, enforcing organizational policies, limiting spending that can spiral when cloud usage isn't controlled and enabling observability of AI systems.
Observability includes automatically generated metrics, traces, logs and transcripts using the OTLP Protocol so that users can check agent behavior in their Redpanda console and take appropriate action such as debugging when necessary.
In addition, the ADP now includes security through the OpenID Connect standard and fine-grained authorization policies so that every interaction with an agent, whether by a human user or another agent, is properly checked and governed.
Meanwhile, the ADP is designed to work in conjunction with any agentic framework so that customers can easily run and govern agents fueled by Redpanda's streaming data capabilities.
Collectively, the new ADP features are valuable for Redpanda customers given that they address numerous concerns and that they add governance to data that fuels real-time analysis, according to Kevin Petrie, an analyst at BARC U.S.
"This announcement … is comprehensive," he said." Redpanda's platform addresses data and AI governance, observability and even FinOps objectives. That's a broader set of capabilities than most platforms have. The announcement [also] stands out because Redpanda is building these capabilities onto a data streaming platform rather than a standard data-at-rest platform."
Perhaps the most significant of the new ADP features is AI Gateway, according to McKnight. Meanwhile, from a competitive standpoint, Redpanda's new governance capabilities could help differentiate the vendor's capabilities from competing streaming data platforms such as Confluent, which is the industry standard for commercial Apache Kafka and is now under agreement to be acquired by IBM, he continued.
Redpanda's streaming capabilities are comparable to Confluent's, outperforming Confluent in some benchmark testing, McKnight noted. But AI governance is where Redpanda could truly stand apart.
"Redpanda is starting to differentiate by shifting from 'data piping' to a dedicated AI governance infrastructure," McKnight said. "Unlike standard tools that require fragmented security at every source, its AI Gateway will provide a centralized control plane for managing policies, token budgets and Model Context Protocol servers."
Petrie similarly suggested that the breadth of the new ADP features help Redpanda distinguish itself from competitors, which include AWS, Google Cloud and Microsoft, beyond streaming data specialists.
"This announcement strengthens its competitive standing by integrating so many capabilities into one streaming solution," he said. "To get the same features from the larger vendors, you would need to buy multiple products. Redpanda also has the advantage of platform neutrality -- it operates across sources, systems and clouds."
Next steps
Following the initial launch of the ADP last October, Redpanda's new governance features represent the second phase of the streaming data vendor's ADP rollout. With an overarching goal of helping customers deploy agentic AI across their organization, more features that enable users to trust and deploy agents are a prominent part of Redpanda's roadmap, according to Akidau.
"We'll be delivering additional features in our AI and MCP gateways, rolling out more agent evaluation functionality, delivering manual and automatic agent kill switches, connecting agents to more data sources, and rolling out [a SQL engine] for federated query support," he said.
One way that Redpanda could better serve its current users and perhaps attract new ones would be to market AI Gateway as a tool for financial governance in addition to technical governance, according to McKnight.
"They could also solidify the 'real-time' advantage by demonstrating that [its] low-latency foundation creates better agents, not just faster data," he said.
Eric Avidon is a senior news writer for Informa TechTarget and a journalist with more than three decades of experience. He covers analytics and data management.
