Google adds TPU 8tx for confidential inference and TPU 8i Max for long context, its fourth TPU system this year

Google added two systems to its eighth-generation TPU line on November 9: TPU 8tx for confidential inference and TPU 8i Max for very long context windows. With TPU 8t and TPU 8i, announced in April, that is four TPU systems in one year, which Google says is a first.

TPU 8TX: CONFIDENTIAL INFERENCE

• Built on the same silicon as TPU 8t and paired with 5th Gen AMD EPYC host CPUs. Google says it keeps data and model weights protected and isolated even while they are being processed, not only when stored or sent.
• Google says it delivers up to 2x lower latency than TPU 8t, with a near-zero performance penalty for the security.
• Aimed at regulated companies and governments: Google says teams can fine-tune on private data, distill models and run reinforcement learning on the same system that serves the model, without moving data between clusters.
• AMD calls it the first cloud TPU system with hardware-enforced confidential computing.

TPU 8I MAX: LONG CONTEXT

• Two TPU 8i chips on one package, giving double the compute, SRAM and HBM bandwidth of TPU 8i, Google says.
• It fits twice the silicon in the same rack footprint. Google pitches it at many agents working across a whole codebase or hours of video at once.
• TechTarget reports 768 MB of on-chip SRAM and 17.2 TB/s of HBM bandwidth.

WHAT WE DO NOT KNOW

Prices and availability dates. Google gave neither for the new systems, and TechTarget reports that TPU 8t and 8i are still due online by the end of the year. All performance figures above are Google's own.