Agentic AI in practice: Lessons from real deployments

Agentic AI promises efficiency, but most enterprises are still figuring out how to make it work in practice.

For CIOs, the stakes are high. When teams deploy agentic AI effectively, it offers critical efficiency gains. When teams fail to do so, projects can stall in the pilot. That tension has driven demand for real-world guidance on what it takes to deploy agents in production.

Martin Bufi, principal research director of applied AI at Info-Tech Research Group, works with enterprises to turn AI use cases into production-ready, agent-driven workflows. In the following Q&A, he outlines where organizations are finding early success, such as invoice processing and IT service management (ITSM), and where they struggle. He also explains why multi-agent architectures, strong governance and code-driven development approaches are essential for scaling agentic AI beyond experimentation.

Editor's note: The following transcript was edited for length and clarity.

What's your role at Info-Tech?

Martin Bufi: My background is in AI. I've been in the space since 2013, starting with robotic systems and then moving into fintech, healthcare and manufacturing. I joined Info-Tech to merge my applied AI expertise with the strategy side. I work with clients to take use case ideas and 'agentify' them -- building enterprise-grade agents for end-to-end workflows and developing prototypes their teams can then deploy in real-world operations.

What patterns are emerging in the deployment of agentic AI?

Bufi: Everyone is raving about OpenClaw and those new frameworks that let users do general tasks. But I focus on enterprise applications, where agents need real guardrails and evaluation. When we build prototypes, we must consider enterprise-wide adoption, identify use cases and figure out how to build agents so an enterprise can plug them into its back-end systems and have them run autonomously.

A lot of the work centers on areas such as HR onboarding, finance and compliance, and inventory analysis and reporting. These involve complex workflows where agents can take on structured, repeatable tasks.

We focus on the non-sexy backend work: long, multi-step processes that pull data from multiple systems and require context and reasoning to complete. We build tailored, process-specific agents -- often using multi-agent architectures to orchestrate the workflow end to end.

From there, we layer in guardrails for safety and evaluation frameworks to test, optimize and support regression over time, because building something is only 50%. Maintenance is the other 50%, and that's what kills a lot of AI projects.

What types of workflows are most targeted for agentic?

Bufi: Invoice management is one of the top use cases. Another one is ITSM. Organizations receive a high volume of tickets from users across the business, and agents can help classify and triage those issues. They can also resolve common problems by checking system status, network health or application availability.

Another area is financial analysis -- for example, reviewing financial proposals or conducting private loan risk assessments. These are tasks that typically require multiple teams to collaborate to produce a final report. We've seen workflows that traditionally take two to three months completed in about a week, with agents handling much of the heavy lifting while humans remain in the loop for review and approval.

Agents today don't operate vertically within a single function -- they operate horizontally across industries. Whether it's financial services, manufacturing or healthcare, organizations share common workflows such as HR onboarding, ITSM and invoice processing.

What challenges do CIOs face when designing workflows for agentic AI?

Bufi: One of the biggest challenges is starting with the workflow itself. In many organizations, processes aren't standardized. Even IT service desk operations can vary across a team -- analysts might all do things differently. Without a clearly defined, consistent process flow, it's very difficult to automate or 'agentify' that work.

The second challenge is moving from a standardized workflow to a multi-agent system. Most people try to get one agent to do the whole task. From our experience, there's never been a single use case that's been solved by one agent. It's always a multi-agent architecture.

This introduces a new layer of complexity around orchestration -- how the agents work together. Depending on the workflow, that could be sequential, parallel or coordinated through patterns that effectively act as logic gates between steps. The right approach depends on the structure and dynamics of the workflow itself, and designing that orchestration is a critical part of making the system work.

Not every step requires the most advanced model.

What challenges do CIOs face when it comes to deploying agentic AI at scale?

Bufi: One is model and cost optimization. Not every step requires the most advanced model, so teams need to balance performance with efficiency and manage FinOps across the workflow.

Another is guardrails. Beyond input and output controls, organizations need agent-level safeguards to ensure agents interact safely with tools and systems and don't delete things, create gibberish or generate content that is not safe for work.

Evaluation is also critical and often missing. A lot of CIOs will tell me they've deployed AI, and I'll ask, 'What do your evaluations look like?' Often the answer is, 'We don't have those yet.' That's a red flag, because without evaluations, you have no way to do regression testing, detect issues or measure whether the agents are actually improving over time.

Finally, there's often a gap between the promise of 'easy' agent-building tools and the reality of building production-grade systems. In practice, many enterprise workflows require more code-driven, engineering-heavy approaches than low-code platforms suggest. We've never actually seen those one-click agents automate full workflows before.

A key goal when customizing is to make agents more deterministic.

What does it actually take to implement agentic AI in the enterprise?

Bufi: It requires a high level of customization. Every organization handles workflows differently. They use different ERP systems, data formats and processes, so there's no true off-the-shelf agent that works out of the box. You can build general agents for things such as HR onboarding or ITSM, but they still need to be tailored to how that specific organization operates.

Another challenge is reliability. Language models are inherently non-deterministic, meaning you can ask the same question twice and get different answers. In an enterprise setting, that's not acceptable. So, a key goal when customizing is to make agents more deterministic -- reducing hallucinations and limiting variability by giving them clear rules and defined ways to handle tasks.

Using languages like Python allows you to build around the full agent lifecycle -- from tooling and workflows to guardrails, evaluation and deployment -- while customizing each step to the organization's needs.

What's involved in building governance around agentic AI?

Bufi: There are a lot of things to consider. First is guardrails -- the first line of defense against bad actors, but also a way to govern how agents operate.

Second is governance around efficiency and FinOps. If you're running multiple agents per workflow, you need to make sure they're not overusing tools or failing unnecessarily. Part of governance is optimizing how those agents run. Then there's quality. We need to ensure agents are producing the right outputs, both in real time and retrospectively -- for example, looking at how they handled a batch of tickets and whether they did it correctly.

That ties directly into evaluation. You can't really have governance without evaluation. It's how you test, monitor and improve performance over time. Access management is another key piece. We don't give agents unlimited access, but scope what they can do. For example, they might be able to query an SQL database but not delete data. They also run in controlled environments so actions can be monitored and, if needed, rolled back.

Finally, it all comes together with observability. We need full traceability into reasoning, tool usage and handoffs so we can understand what the agents are doing and why.

Have you noticed differences in adoption approaches between industries or company sizes?

Bufi: Yeah, adoption really comes down to the ability to buy versus build. Many organizations want to adopt agents but don't have the technical teams to build them from scratch. So, they default to a 'buy' approach, often aligning with a specific vendor -- for example, Microsoft -- and end up relying on tools like Copilot to build agents.

But for more complex workflows, Copilot has limits. It's primarily built for assistance use cases, and you can only get so far with it. At some point, organizations must decide whether a workflow is complex enough to justify a more code-driven approach.

When they make that leap, it usually means adopting a broader technical stack -- using platforms such as Azure AI Foundry or OpenAI -- and rethinking how they build and deploy agents. It often leads to creating dedicated teams focused on agent automation -- similar to what we saw with robotic process automation -- along with upskilling efforts to support those workflows.

You wouldn't believe how many organizations don't have clearly defined processes.

What we're seeing is a shift in adoption. Companies can only experiment for so long. If early approaches don't deliver results, they start looking for alternatives. When it's done right, the ROI can be significant, but getting there is still difficult, which is why there's so much failure in the market today.

What best practices would you recommend for CIOs considering agentic AI?

Bufi: First, standardize your workflows. You wouldn't believe how many organizations don't have clearly defined processes. I need to be able to start that job tomorrow following the process flow.

Second, understand the business value. Too often, CIOs want to automate or use AI without clearly defining why. They don't always have the underlying data -- things like ticket volumes, process inefficiencies or baseline performance -- that allow them to measure affect. Without that, it's hard to compare an agent-driven process to the current state and demonstrate improvement.

Third, define KPIs upfront. Many organizations haven't identified the metrics they want agents to improve. You need to start with a clear problem, understand the inefficiencies in numerical terms and establish the KPIs you're trying to move.

How do you see the role of agentic AI evolving in enterprise IT over the next few years?

Bufi: The frameworks will continue to mature, and we'll see more studies and case studies on what works and what doesn't. You'll still have part of the market doing more citizen-level agent development for smaller workflows. However, enterprise IT leaders will realize they need dedicated teams to build and manage more complex, long-running systems.

Humans will remain in the loop. Use case owners aren't being replaced, but they're shifting into roles where they facilitate and manage the agents. At the same time, organizations will build agent-focused teams that develop new use cases and maintain them over the full lifecycle. No AI project is ever really finished once it's built -- that's just the beginning.

More companies are starting to upskill their technical teams to take a use case, automate it with agentic AI and then continuously manage and improve it over time. It becomes a collaborative model: agent teams working closely with business SMEs. There's no real way to remove that partnership -- it's a shared responsibility.

Tim Murphy is site editor for Informa TechTarget's IT Strategy group.