TORONTO -- At a recent machine learning conference, attendees were asked to raise their hands if they were starting to work on AI agents. Across the room, many hands shot up.
This dynamic was indicative of the general feeling about AI agents throughout the Toronto Machine Learning Society Summit: this form of generative AI, first widely popularized last year, is becoming tangible for developers and engineers to work with.
You're going to see use cases of agents adding real value, which is something that is happening a lot more than it used to.
Chris AlexiukCo-founder and CTO of AI Makerspace
Chris Alexiuk, co-founder and CTO of AI Makerspace, an educational community for building large language model (LLM) applications, was an organizer of the conference's AI agent sessions. In an interview with Informa TechTarget, he said that when choosing sessions, he wanted to highlight AI agents' increasing usability across technical levels.
"You're going to see use cases of agents adding real value, which is something that is happening a lot more than it used to," Alexiuk said.
The conference, held earlier this month, highlighted many case studies of agentic AI applications, such as agents for data management, agentic AI governance frameworks and self-hosted agents. To capitalize on their usability, developers building AI agents turn to hands-on development for unique needs.
Technical environments lead applications
AI agents are a form of generative AI that reason and dynamically perform tasks to achieve goals with little to no human intervention.
"They can write code," said Niels Bantilan, chief machine learning engineer at AI orchestration platform Union.ai, in an interview. "They can make API calls. They can use tools in various forms."
Greg Loughnane, co-founder and CEO of AI Makerspace, echoed this in an interview, saying that technical tasks like coding are becoming a key area for AI agents.
Software engineers work in self-contained systems, Loughnane said. Therefore, AI agents in technical environments can offer high value -- building features, completing tickets and fixing bugs -- while encountering less complexity than other job functions.
"It's like a proving ground for how you can imagine these agentic systems in other domains," Alexiuk added. Seeing agents' utility in software development can help show other industries the benefits of having an autonomous agent or team of agents to help get tasks done.
As opposed to creating new use cases, agentic use often arises from existing needs or growing projects.
"Agents do organically grow out of increased scope," Alexiuk said. For instance, what could have started as a retrieval-augmented generation (RAG) application in 2023 might be working toward an agent-driven application today.
Many conference sessions highlighted such use cases. In a session on the value of agentic AI tooling for data needs, Josh Goldstein, a solutions architect at open source vector database vendor Weaviate, explained how AI agents trained for data tasks can help teams make sense of complicated data needs in RAG systems.
"A lot of data is messy," Goldstein said. Agentic RAG agents can help by acting as data experts to retrieve and aggregate important data.
Weaviate currently has three data agents: a query agent and a transformation agent, both used for execution and tactical tasks, and a personalization agent that captures user behaviors to create real-time recommendations.
These data agents work across the domain, Goldstein said. They can perform many operations when working with a vector database or RAG system, such as querying and augmenting data to create personalized user answers.
Josh Goldstein, a solutions architect at Weaviate, highlights at Toronto Machine Learning Society Summit how the company's data agents can help answer complicated data queries.
The role of agentic AI governance
Like any generative AI technology, AI agents have their fair share of risks. Bantilan cited the threat of hallucinations and bad actors taking advantage of security vulnerabilities as top concerns.
Other risks unique to AI agents stem from their differences from other technologies. For example, an AI agent can turn a defined 10-step project into hundreds of steps, depending on how the agent divides subtasks.
Agents can be in control of many project steps, Bantilan, of Union.ai, said. "This is where the magic is, but also where a lot of the failure modes may happen."
For example, agents can get stuck in an infinite loop while trying to reason through a problem, he said. There can also be a mismatch between task and agent, leading to overallocation of tokens or compute resources.
AI agents can also access tools directly, which adds complexity to hallucinations, Bantilan added. An agent could use the wrong tool for a given task, leading to incorrect outcomes.
Due to these concerns, agentic AI uses today are low risk with a tight loop with a human, Bantilan said. For instance, tasks such as coding and research assistance are more amenable to human verification than complex applications, such as an AI agent that would interact directly with a medical patient.
Organizations implementing AI agents also turn to explicit governance frameworks to mitigate risks. In another session, Ankit Misra, Canadian bank CIBC's director of AI governance, highlighted how organizations can reduce AI agents' risk exposure.
AI agents have a unique portfolio of risks, Misra said. That means agents need a new kind of governance model.
For CIBC, this means governance by design. Misra noted agentic governance best practices such as designing with risks in mind, collaborating with stakeholders and continuously monitoring workflows.
"Compliance shouldn't be an afterthought," Misra said. "It should be guaranteed by the architecture of the system itself."
Ankit Misra, director of AI governance at Canadian Imperial Bank of Commerce, explains at TMLS Summit CIBC's six key tenets of governing AI agents by design.
Considerations when building AI agents
While AI agents might feel new, they are a natural progression of the technology that came before, Bantilan said. AI agents are built on the familiar functions, constructs and primitive building blocks all programmers can access.
Bantilan said engineers need to remember the basics when building agents: defining objectives, success metrics, raw materials and data.
"I care a lot about data quality and data," he said. LLMs are trained on all different types of data, and therefore, substantial effort is needed to curate data sets to train AI agents.
Another top consideration when building agents is the tradeoff between performance and cost, Bantilan said. Once agents are in production, costs can easily balloon due to scaling and orchestration needs.
Bantilan recommends starting with LLM providers that can be called through an API. When agents become more advanced and production-ready, organizations can consider controlling costs -- for instance, by self-hosting models.
In a session about self-hosting for AI agents, Amanda Milberg, principal solutions engineer at Doubleword, an inference platform for self-hosting AI, took attendees through a demonstration using Open WebUI and qwen3:1.7b to self-host an agentic workflow.
"Self-hosting an agentic system can act as a strategic advantage," Milberg said. For one reason, agents need significant amounts of tokens. Especially with recent reasoning models, it can be difficult to pinpoint how much agentic use will cost organizations. In comparison, self-hosting is often a fixed cost.
Second, owning the entire AI stack comes with technical advantages, she said. One such advantage is prefix caching: a technique that stores common inputs so the LLM doesn't have to compute each time. This is beneficial for cost, latency and performance.
Self-hosting also enables teams to fine-tune agentic models to their use cases, Milberg said. And lastly, organizations can control compliance more easily because everything stays within their security environment.
Amanda Milberg, principal solutions engineer at Doubleword, queries her self-hosted agentic system to perform a know-your-customer check at TMLS Summit. The agent relies on specific tools it can access to perform the check accurately.
Use a build-first approach, and remember domain expertise
Bantilan suggested that developers start by building an agent locally, from scratch, on their computer. "You'll learn a lot about the limitations of these systems, where the engineering and the gluing of different inputs and outputs need to happen."
That's not to say frameworks like LangChain and CrewAI aren't useful, Bantilan said. They can be great tools for developers who need results fast. But all frameworks have their limitations.
"They have a happy path, and once your requirements veer off that happy path, you're on your own," Bantilan said. For that reason, developing agents independently can help developers gain the skills to customize when frameworks alone are insufficient.
Alexiuk and Loughnane shared a similar sentiment, recommending a "build-first" approach to learning the foundations of agent development. From there, teams can consider custom-building AI agents that fit their use case or area of expertise.
Developers should also emphasize their unique subject-matter expertise and data, Loughnane said. The better teams can understand the details of how their data will contribute to decision-making, the more successful an agent will be.
"Everybody has the agents," Loughnane said. "What they don't have is your data … your customers [or] your ability to understand your business operations."
Olivia Wisbey is the associate site editor for SearchEnterpriseAI. Wisbey graduated from Colgate University with Bachelor of Arts degrees in English literature and political science and has experience covering AI, machine learning and software quality topics.
