Extending its eighth generation of TPUs, Google has added two new variants, the 8i Max and 8tx, which specialize respectively in agentic computing and confidential computing.
Google created its TPU (Tensor Processing Unit) line of hardware accelerators to meet the specific demands of its AI work and, increasingly, that of its cloud customers. Each new generation of TPUs has addressed the particular bottlenecks for AI at the time.
“[AI] used to be mostly about how much compute you had available, but now it's also about how quickly data can move, how efficiently you can use the memory, how do you minimize latency, and how you manage the cost and security of running all these workloads at scale,” said Mark Lohmeyer, Google Cloud’s vice president and general manager for AI and computing infrastructure, in a press briefing last week.
A TPU for agentic work
Google designed the TPU 8i Max for supporting massive context windows required for agentic work.
“With roughly two-thirds of AI compute now going to inference, specializing [in] inference hardware is the right call,” wrote Patrick Moorhead, founder, CEO and chief analyst of Moor Insights & Strategy, in an email.
Google designed the 8i Max to give agents as much memory as possible, mostly by adding a second TPU onto a multi-chip package. A group of AI agents may work together on tasks such as developing software or creating a video. Thanks to a two-chip design, the 8i Max gets 768 MB of on-chip static RAM and 17.2 TB/s of high-bandwidth memory (HBM) bandwidth.
When used in large numbers, these TPUs can also maximize the amount of compute per square foot of the data center.
Many TPUs can collectively work on bigger problems. New software Google recently open-sourced, called TPU Sync, orchestrates chip HBM, host DRAM, shared storage and the data center network into a single memory hierarchy.
Securing the workload
The TPU 8tx focuses on secure and confidential inference.
Secure AI computing is one emerging class of workloads the company is finding from customers. Google built the TPU 8tx for sensitive government work and enterprises in regulated industries.
“A bank or government agency can fine-tune, distill and run reinforcement learning on the same confidential system that serves the model, without moving sensitive data between clusters,” Moorhead wrote.
TPU 8tx enables teams to fine-tune models using their private data, distill large models into smaller ones, and train their agents through reinforcement learning, all without migrating sensitive data between separate clusters, according to a blog posted Monday by Amin Vahdat, senior vice president and chief technologist of AI and infrastructure at Google.
Data and model weights remain encrypted and isolated at the hardware level during processing, with a near-zero performance penalty and half the latency than the TPU 8t, Lohmeyer said during the briefing.
An increasingly diverse user base
As with previous generations, Google designed the TPUs with the input of the Google DeepMind research lab. But with each generation, the company is building a base of outside users to consult with, both from its cloud service and channel sales.
More than 100 customers and partners – including frontier labs, trading firms, retailers and universities – are now using TPUs, according to the company. One is Anthropic, which will expand its use of next-generation TPUs beginning in 2027. The California Institute of Technology has shifted some of its own cancer research over to Google Cloud TPUs from its own HPC system, Moorhead learned in a conversation with the team in charge of that operation.
Google did not provide any availability dates and pricing for the TPU 8i Max and TPU 8tx. The 8t and 8i are still due to go online by the end of this year.
For Moorhead, these new TPUs represent a turning point in the direction of AI computing. No longer is the GPU the only focal point.
“Agents put the CPU and host platform back at the center,” Moorhead wrote.
Hardware specialization
Google launched the first batch of eighth generation TPUs in April, with two variants. The 8t variant is for large-scale training, and the 8i is for inferencing. This generation of TPUs was the first to bifurcate the AI workload into two separate specialties, each with its own modifications to hardware, Lohmeyer said.
Key to the TPU architecture has always been a rack-level design, rather than just innovating at the chip level itself. The eighth generation is the first to apply this specialization to specific workloads, collectively optimizing CPUs, system DRAM, networking and shared storage for the task. The hardware specialization allows the 8t to offer up to 2.7 times better training performance and the 8i to provide 80% better inference price-performance compared to previous TPU generations, according to Vahdat.
"You can't optimize that infrastructure just by focusing on the chip,” Lohmeyer said.
The new TPUs continue this design practice, addressing the current bottlenecks of AI operations.
Freelance news writer Joab Jackson has been writing about back-end IT technologies for the past three decades. His grandfather programmed mainframes, and his father wrote computer games for hobbyist programming magazines in the 1980s.
