Blue Planet Studio - stock.adobe
Harness Agent DLC targets AI agent development gaps
DevSecOps for AI agents requires more than code review -- Harness beefs up behavioral and security controls and hints at a possible observability expansion.
Development tools to create agents abound, as do agent orchestration tools to govern them in production, but the workflow between those two stages is less well established.
DevSecOps platform vendor Harness shipped Agent DLC this week, focused on that portion of the AI agent development lifecycle between creation and production deployment. Agent DLC builds in tools to evaluate and test probabilistic agents that traditional CI/CD pipelines lack, where development teams can watch and score the behavior and responses of AI agents with greater nuance than the simple "pass/fail" of traditional code reviews.
"In historical software, you see a crash, you can look at that log, you can look at that trace, and you can fix it," said Trevor Stuart, senior vice president and general manager at Harness, in an interview with TechTarget. "A failure mode for an agent is hallucination. It tells you what's happening, but maybe it's not actually accurate. How do you test something that's non-deterministic?"
Harness Agent DLC has the potential to answer important questions facing enterprise developers building agents, said Torsten Volk, an analyst at Omdia.
"Dev teams have been trying to figure out how to get traditional deterministic code and fancy new probabilistic AI capabilities through the CI/CD pipeline," Volk said. "I remember conversations about exactly this topic where people were asking me: 'Is this realistic? Will it ever happen?'"
Harness Agent DLC adds five new features to the existing Harness DevSecOps platform:
- Harness AI Evals to perform quality tests that score agent performance based on non-deterministic failure modes such as hallucinations.
- Agent deployments to deploy agents to cloud-based environments such as Amazon Bedrock Agentcore and Google Agent Runtime with support for canary releases, approvals and Open Policy Agent governance as code.
- AI Configs to support the management of prompts and large language model (LLM) changes alongside agents, with support for feature flagging.
- AI Asset Catalog to associate agents and their artifacts with code repositories and owners within an organization.
- Harness AgentTrace, which uses OpenTelemetry data collection to observe single-agent runs and troubleshoot performance slowdowns.
By managing agents alongside code and making models "explicitly configurable," Agent DLC "allows users to swap models as appropriate for their needs and price tolerances," said Andrew Cornwall, an analyst at Forrester Research.
"AI evals mean that models can be evaluated in the same way as other assets, tested and rolled back automatically," Cornwall said. "That means you can return to a known good state if you discover a model in production starts swearing at users or giving away cars."
The Harness AI Asset Catalog could also potentially help with a growing pain point for enterprise AI: managing all the AI assets, APIs, MCP servers and tools that developers are deploying, Cornwall said.
"Many organizations have encouraged experimentation with AI, but when these experiments start getting included in production releases, businesses need to know what they're depending on," he said. "This catalog integrates with the Harness IDP, so hopefully developers will also be able to discover what's already out there and do less duplicate work."
Observability and security round out Agent DLC
AgentTrace isn't a standalone observability product for Harness, but rather a way to tie agent performance data into its broader platform. Stuart didn't rule out future expansion of Harness observability tooling.
"While Harness is not entering into observability, which will be a very big step for Harness when we do, given our founder [Jyoti Bansal]'s background as the founder of AppDynamics, we are beginning to collect traces to inform our products," Stuart said. "AgentTrace will look at and pull down all of the tool calls, everything that the agent did, and it will allow us to inform products like AI evals, inform some of our security products, as more of a platform capability."
Similarly, a set of new security features will be built into the Harness platform to shore up DevSecOps workflows for AI agent development. They include support for LLM scanning in Harness Static Application Security Testing and Security Testing Orchestration modules; support for AI bill of materials in its Supply Chain Security tool; and agent-specific pre-deployment security tests.
Post-deployment, a new Agent Discovery feature will detect agents and map their connections, while AgentTrace provides an audit trail of prompts, reasoning, tool calls and outputs. An enforcement feature in development by Harness will shore up security posture management for AI agents in production, according to a company blog post.
Stuart did rule out one area of possible expansion for Harness in AI agent development: it has no plans to add its own agent builder tool, he said.
"They're kind of equivalent to coding tools," Stuart said. "Our goal is to help you ship that code to production safely and securely."
Harness competitors offer parts of what the vendor has added to Agent DLC. There are also direct competitors in AI agent development tools that include agent builders, such as n8n, Gumloop, CrewAI, AWS DevOps Agent release management and Google's Gemini Enterprise Agent Platform. JFrog, partnered with GitHub, offers similar supply chain and agent security features, while GitHub and GitLab DevSecOps platforms include AI agent governance and observability features. Numerous IT security vendors offer AI agent security tools, and Palo Alto Networks is building an observability platform based on its acquisitions of Chronosphere in November and Embrace this week. OpenAI launched its Presence AI agent development platform in limited general availability this week, focused on deployment workflows for voice and chat agents that handle customer-facing and internal applications.
"I do not think the individual features are especially differentiated in isolation. There are already numerous vendors offering agent discovery, red teaming, runtime firewalls, tracing, model scanning or tool governance," wrote Katie Norton, an analyst at IDC, in an email to TechTarget this week. "The more differentiated part of the Harness proposition is the attempt to give all those functions a common platform context -- especially within the platform used for development."
Beth Pariseau, senior news writer for Informa TechTarget, is an award-winning veteran of IT journalism. Have a tip? Email her or connect on LinkedIn.