Microsoft on Tuesday upgraded Azure's infrastructure to speed up and contain AI agents, while giving companies more control over where their AI runs.
At Microsoft Ignite, the vendor focused on core services to isolate agents in sandboxes and make them more efficient. The sandbox is wrapped by two services: Agent Grid, a new service that controls network resources agents can reach, and ContextCache, which saves an agent's work so it doesn't have to start over on a new server.
"Agents are insider risks. We need ways to isolate them and control them," said Mark Russinovich, CTO and deputy chief information security officer at Microsoft Azure, during a keynote speech at Ignite.
Microsoft also gave a twist to the ongoing build-vs.-buy debate over whether companies should self-manage AI or rent it as a service. Microsoft is letting companies plug self-managed Azure Kubernetes Service (AKS) clusters into Foundry, its managed AI service.
"You can do kind of the middle where you can have Foundry managing things on top of AKS," Russinovich said.
These and other new features launched at Ignite are pushing Azure into what Russinovich called "one connected system, from the silicon all the way up to the platform."
Microsoft and its competitors, Google and AWS, have invested billions in cloud infrastructure to build faster and more secure systems. The battle in the high-stakes AI market is also about acquiring and retaining customers.
These new features are steps toward those goals, but "the other guys have similar capabilities in place," said Jack Gold, principal analyst at J.Gold Associates. "No hyperscaler stays ahead of the others for very long."
CIOs need to get ahead and start experimenting with the features, which are in preview, said Sanchit Vir Gogia, CEO of Greyhound Research.
"Platform teams should pilot [the new features] now and hold production commitments until the documentation and service terms arrive," Gogia said.
Faster microVMs
Agents are deployed in mini-computers called microVMs, which lock down agents and keep them from harming outside systems. Russinovich claimed Azure can load thousands of microVMs – which start on demand and then shut down when not needed – in less than half a second.
Google and AWS have similar microVM offerings to lock down agents.
"This capability really should be seen as a minimum agentic infrastructure requirement," Gold said.
Agent Grid adds a layer of protection around microVMs by controlling the resources agents can access over a network.
"If it can't access a certain website or you want it to access only a subset of websites, you can enforce those policies right there in the network right next to the agent sandbox," Russinovich said.
Isolation stops agent-written code from escaping onto the host, but "it does nothing about an agent misusing access it was legitimately given," Gogia said.
ContextCache, which Microsoft first showed earlier this year and detailed at Ignite, saves the context of a long-running agent so it can be reused if reloaded on another server. For example, long-running agents on GPUs will reuse context instead of recomputing it.
"We need to be able to feed these models so that they ... save money by not having to recompute work," Russinovich said.
ContextCache "is an important capability, as shared ‘knowledge’ between agents and agent runs helps keep the costs of processing down, as well as speed up the output results," Gold said.
Microsoft talked up savings from ContextCache but didn't provide a figure. For comparison, OpenAI discounts cached input by up to 95% and Anthropic by 90% or more, according to Greyhound Research.
"Those discounts apply only to the repeated part of the input, never to the answer the model writes," Gogia said.
Plug Foundry into homegrown clusters
Microsoft is providing a new feature for enterprises to connect Foundry, which companies rent, to self-managed AKS clusters.
Customers will now be able to deploy open-weight models and agents from Foundry to their own AKS clusters, Microsoft said in a statement.
In Russinovich’s demonstration of how the system works, a platform engineering team set up an AKS cluster with security controls and open-source components. Microsoft's Foundry service deployed a Llama 3.3 70B model onto the cluster, and an agent was built on it to diagnose a machinery issue. A GPU node on the factory floor with a camera was added to the cluster through AKS flex nodes. Rather than sending humans, the agent diagnosed the issue in four minutes instead of two hours.
By bringing Foundry to a company's AKS clusters, Microsoft is bridging the gap in the build-or-buy debate.
"[It] ends a false choice between a fully managed service and a do-it-yourself stack," Gogia said. "The new middle path lets a team start managed and move onto its own clusters without changing tools."
Avoid vendor lock-in
The new features may look attractive in the build-vs.-buy debate, but CIOs should maintain diversity when it comes to cloud vendors, analysts said.
Specific tools on Azure can lock down customers just like on AWS and Google, and then businesses need to discuss returns and find an optimum solution, Gold said.
CIOs should buy what removes operational work their organization has little reason to own, and build where requirements or business advantage justify maintaining it, said Stephanie Walter, practice leader for the AI stack at HyperFrame Research.
"Before committing, establish what it would take to move the agent's state, tools, permissions and evaluations elsewhere. Model choice alone doesn't make an application portable," Walter said.
But wait, there’s more
In addition, Microsoft launched Azure Catalyst, which modernizes legacy on-premises apps in the cloud with AI agents.
"Azure Catalyst takes a bunch of components that we've already had and brings them together with agents, completing this flow from end to end," Russinovich said.
Microsoft also launched new VM instances that support confidential computing features on newer Intel and AMD chips. The instances create a black box around data and applications, and require attestation before outside applications are granted access.
Agam Shah is a freelance writer. Reach him on LinkedIn at linkedin.com/in/katroo.