Google's new Agent Compute Engine (ACE) includes microVM isolation for AI agents, giving cloud-hosted infrastructure an advantage with some IT pros amid a growing build-vs-buy debate over enterprise AI.
ACE, launched Oct 22, isolates and manages agents, each with its own compute resources called microVMs, which are lightweight, hardware-isolated virtual computers.
Locked-down agents wake up when an AI model or user responds, and go back to sleep when idle. Google claims microVMs are more efficient than regular virtual machines, while offering host-level isolation for agentic security.
"While traditional compute primitives are excellent for standard workloads, they weren't optimized for the unique, highly iterative nature of AI agents," according to a Google blog post.
ACE already runs Google's Gemini agents, and a customer preview is due in a few months. Pricing won't be disclosed until the service becomes generally available, according to a Google spokesperson.
The news comes on the heels of multiple new product rollouts from IT infrastructure vendors offering agentic security that operates within infrastructure, outside agents and their orchestration software, following high-profile reports this year of 'rogue agents' breaking out of their sandboxes.
Broadcom added agentic security to its VMware hypervisor with HyperArmor on Oct. 13; Nvidia and partners unveiled the OpenShell Reference Architecture on Oct 21, which includes hardware-based agentic security features; IBM this week added agentic security features to infrastructure automation tools such as Vault and Terraform, including support for hardware-rooted cryptographic identity attestation.
Some of these vendors are promoting products, including hardware and infrastructure software, that enterprise buyers manage themselves in a hybrid or private cloud, playing to concerns about security, intellectual property and digital sovereignty as enterprise AI emerges.
ACE builds on Google's Gemini Enterprise, adding security, management and observability features traditional VMs lack, said Jack Gold, principal analyst at J. Gold Associates.
Those are key considerations in establishing operating models that identify, secure and manage hundreds or thousands of agents, Gold said.
While Gemini Enterprise can run on-premises, its management plane is hosted in the cloud, adding to the choices available to enterprises as they weigh whether to use a managed service like ACE or build their own internal system, Gold said.
"Deploy-your-own versus deploy-in-the-cloud comes down to speed of deployment, available infrastructure, security and resources," Gold said. "This is much the same discussion about on-prem software deployment vs SaaS."
How ACE runs agents
Once agents execute code and use tools, enterprises need somewhere to contain that activity, said Stephanie Walter, practice leader for the AI stack at Hyperframe Research.
"A hardware boundary helps contain execution, but it doesn't prevent an authorized agent from making a bad change," Walter said.
ACE gives each agent its own identity and locks it inside a microVM with secure storage and controlled network access. Agents work concurrently, sharing unchanged copies of the OS and programs in memory.
For consulting firm 28Stone, which builds technology for capital markets firms, ACE could change how agents are provisioned, said Venu Bolisetty, global head of AI and innovation at 28Stone.
"Isolating thousands of agents and pausing them while they wait took Google custom silicon and a purpose-built platform, and few trading firms gain anything by rebuilding that,” Bolisetty said.
Google's ACE brings a fresh approach to the microVM process, Bolisetty said.
In evaluating the technology, 28Stone pointed to ACE's zero-credential sandboxes as a key feature. The sandboxes contain no passwords, keys or other credentials. Each agent is assigned a temporary ID that requires verification from a Google gateway to access other systems. Credentials are added by the gateway after it checks each request, and the sandboxes have no direct access to the internet or the company network.
"Today, agents raise the same question [of where workloads should run] but now with higher stakes, because they change systems as well as read from them," Bolisetty said.
By comparison, AWS and Azure, by default, let agents use credentials inside sandboxes. The credentials are handed to them inside the sandbox, while Google doesn't store credentials inside sandboxes.
Like Azure's newest sandboxes, ACE can snapshot a running agent and branch it into many parallel sandboxes almost instantly, and rewind and roll it back.
In its blog post, Google said ACE's branches share memory, so it can run thousands of them without duplicating resources.
"It wakes a paused agent faster than a person would notice, so the infrastructure is not the constraint," Bolisetty said.
But "what ACE doesn't provide is everything specific to a firm's desk," such as connections to trading systems, compliance-approved policies and workflows like broker evaluation, he said.
ACE adds hardware-based isolation as a service
ACE is like purchasing a service from Google Cloud, but under the hood it adds some features native to Google's AI chips, called TPUs, Gold said.
"It's not an issue of running on particular hardware like [Nvidia’s] Vera Rubin," but rather "the infrastructure that Google has put in place," Gold said.
The hardware and software architecture matters more for enterprises weighing on-premises agent systems built on Vera Rubin, and they should "start by separating model inference from agent execution," Walter said.
Nvidia's reference architecture uses Rubin for inference and Vera CPUs for agent harnesses and tools. ACE supplies a managed execution environment, Walter said.
The platforms can be alternatives for parts of a deployment or even complement each other, depending on where models, tools and data reside.
"A company does not need to buy an entire Vera Rubin system simply to run agent sandboxes," Walter said.
When to buy or build
Managed infrastructure is attractive when demand is unpredictable or a team doesn't want to build and run AI systems, Walter said.
“On-premises deserves consideration when workloads must stay close to sensitive data or local systems," Walter said.
Most 28Stone clients run agents both in the cloud and on-premises, placing them by workload type rather than agent by agent, Bolisetty said.
The placement comes down to three things: how sensitive the data is, where it already lives and what the agent is allowed to do.
A research assistant summarizing filings and market data already in Snowflake belongs in the cloud. An agent that amends client data in an on-premises system usually stays internal, Bolisetty said.
"ACE settles one part of the debate: the secure environment that agents run in is now something to buy rather than build," Bolisetty said.
Trading firms rarely build their own trading software but invest in the integration and workflow around it, and “agent infrastructure should follow the same split,” Bolisetty said.
Buying ACE makes the most sense for interactive, "bursting" workloads, such as in retail or medical research, or when an enterprise's security engineering team is overloaded, said Yousuf Khan, CEO of AI startup Clario.
ACE assumes attackers could take over the sandbox, so credentials are kept outside and requests go through a gateway.
"Even a root-level compromise finds no secrets,” Khan said. "Rebuilding that, plus hardware attestation, patching, and red-teaming, is a serious platform investment."
If an enterprise's data and models already live on Google, it makes sense to keep agents in ACE, Khan said.
Building internally may be better for sustained, predictable workloads, or when data sovereignty is hard to manage.
"If Google can aid this, it's a winner," Khan said.
Latency and vendor lock-in
On-premises AI systems remain necessary for low-latency operations, but AI speed also matters.
Cloud systems can complete trades in microseconds, but 28Stone would still keep AI agents off the execution path, Bolisetty said, adding, "The reason is the model, not the cloud."
Trading systems are built for consistent latency, while a model's response time can swing by seconds, Bolisetty said, adding "ACE itself adds very little to that wait."
ACE could handle other time-sensitive work, such as pre-trade research, algorithm monitoring and trade analysis, Bolisetty said.
ACE could also lock some customers into Google Cloud, just as using similar tools on Azure or AWS can.
“For lock-in, examine how identities, policies, workspaces, and traces would move elsewhere,” Walter said.
Firms that want more control or need to stay portable across clouds can run Agent Substrate, Google's open source equivalent to ACE, on their own Kubernetes clusters, Bolisetty said.
Comparing cloud and on-premises costs and control
On-premises AI infrastructure gives enterprises more operational control, but also responsibility for patching, capacity, availability and the hardware, Walter said.
No single utilization threshold makes owning the infrastructure cheaper, Walter said. It depends on the workload, the full cost of operating the system and the price of an equivalent cloud service.
Comparing GPU utilization with the price of an agent sandbox mixes different resources. Enterprises should instead define an equivalent service, including inference, execution, storage, networking, security and staffing, then measure cost per correctly completed task at the required response time, Walter said.
ACE becomes attractive if its pricing reflects idle periods and avoids enough operating work to justify the premium, Walter said.
An agent waiting for a model response or human approval should not need the same active resources as one executing code.
"Ownership needs sustained, useful demand, not just busy hardware," Walter said.
Agam Shah is a freelance writer. Reach him on LinkedIn at linkedin.com/in/katro
