Compute#AI chips#Model training#Model inference
Google open-sources TorchTPU to run PyTorch on TPUs
Google open-sourced TorchTPU under Apache 2.0 on October 20. Google says PyTorch code needs minimal changes to run on TPUs. For teams cutting Nvidia reliance.
TorchTPU is Google’s native PyTorch software stack for its Tensor Processing Units, co-developed with the PyTorch team at Meta. On October 20, 2026, at the PyTorch Conference 2026, Google said TorchTPU is now open source on GitHub under the Apache 2.0 license. Google says existing PyTorch workloads need only minimal code changes to run on TPUs. Its developer blog is more specific: a training script needs only minimal changes to its device initialization code, and the native backend is selected with torch.device("tpu").
Key points
- Release and license: the google-pytorch/torch_tpu README and LICENSE text both say Apache 2.0. GitHub shows the license as “Other” because the LICENSE file begins with an extra
Files: ./*line. The torch-tpu 0.2.0 release on PyPI went up on October 19, a day before the announcement. - Code changes: Google says existing workloads need minimal code changes. Its blog recommends torch.compile to reach production-level performance, and the developer blog says most models need a sequence of configuration changes to reach good utilization.
- Performance: Google reports up to 52% model FLOPs utilization (MFU) on Qwen 3 8B when training through TorchTitan, across up to 8,192 Ironwood TPUs. These are vendor-reported figures, and we found no independent test.
Background
Google had already described the design and a 2026 roadmap in its developer blog on April 7. The roadmap listed “the launch of our public GitHub repository” as upcoming work. The repository was created on March 9, but we could not establish when it became public.
Before TorchTPU, developers wanting PyTorch on TPUs had two options, according to Google’s developer blog: rewrite the code in JAX, or use the older torch_xla path. Google now presents TPUs as a native PyTorch device.
In the same week, Amazon released TorchNeuron, a native PyTorch backend for Trainium chips, on Monday, October 19, and TechCrunch wrote the two stories together. Google says users can train, serve and deploy on TPUs with the tools they already rely on, including vLLM, SGLang, llm-d, TorchTitan and Ray. The vLLM backend was developed with Inferact, the company behind vLLM, and the SGLang backend with RadixArk. llm-d 1.0 shipped on October 20 as well, and our llm-d 1.0 report covers it. In September, Inferact published its own TPU inference test, covered in our Inferact report. Those numbers belong to Inferact, not to TorchTPU.
The facts
The table lists the figures we checked. Each comes from Google, the repository, PyPI or a partner, and its conditions sit in the same row.
| Item | Figure | Source and conditions |
|---|---|---|
| License | Apache 2.0 | README line 231 and the LICENSE text; GitHub shows “Other” because of the extra Files: ./* line |
| PyPI 0.2.0 | Uploaded October 19, 2026 | Wheels for Python 3.11 to 3.14, all Linux x86_64, no source distribution |
| Qwen 3 8B training | Up to 52% MFU | Google Cloud blog; TorchTitan training path across up to 8,192 Ironwood TPUs |
| Llama 3 405B training | About 57% MFU | Google Developers Blog; TPU v7x, 128 to 1,024 chips |
| Qwen3 235B (MoE) training | 32.42% on one 1,024-chip slice; 30.51% on 8,192 chips | Google Developers Blog; TPU v7x |
| Inference throughput (Zenteiq) | Over 4x higher | Partner figure quoted by Google; llm-d routing on GKE, per 1,000 TPU v6e |
MFU measures how much of a chip’s peak arithmetic goes into the model’s own computation. These figures come from particular models and cluster sizes, so another workload may not match them. The developer blog makes the same point: for most models, reaching a good MFU on TPU takes a series of configuration changes, not new code.
Eager mode is the easiest place to start, but it is not a performance setting. The developer blog says so directly: every operation becomes its own program, and the TPU sits idle between launches. Use it to bring a model up and debug it, then move to torch.compile or TorchTitan’s Graph Trainer. Google’s blog describes the same sequence.
What others say
TechCrunch’s Julie Bort describes moving an app to TPUs as a simple configuration change with no rewrite. She adds that none of this means developers will automatically drop Nvidia hardware at scale. SiliconANGLE’s Mike Wheatley describes a few simple configuration tweaks, and quotes Holger Mueller of Constellation Research. Mueller says rebuilding PyTorch makes migration easier, but how far the two cloud providers can chip away at Nvidia’s position “remains to be seen.”
The outlets and Google do not describe the effort the same way. Google’s own phrase is “minimal code changes,” and the developer blog narrows it to device initialization code. TechCrunch and SiliconANGLE both describe configuration-level changes. We use Google’s wording, because it comes from the primary source.
On Hacker News, the repository post had 327 points and 74 comments when we read it. Two practitioner comments are worth noting. One user said their team evaluated TPUs but decided to “stick with Nvidia, at least for one more generation.” Another recalled having to fight XLA to get PyTorch running on TPUs, and welcomed first-party support. Both are individual experiences, not a survey.
Our take
For teams already training or serving on Google Cloud or their own TPUs, the practical change is that PyTorch now has an official native path, so the code does not have to be rewritten in JAX. Teams that only use Nvidia GPUs or call hosted APIs will see no direct effect.
Three questions remain open. First, independent benchmarks: every throughput and utilization figure here comes from Google or its partners, and we found no third-party reproduction. Second, maturity: PyPI lists version 0.2.0, and the VeRL TPU backend and the TPU agent suite are still in preview. Third, pricing and regional availability, which neither Google post nor the README covers.
If you want to evaluate this, start with a small model in eager mode on a TPU VM, then measure performance separately. Do not budget from Google’s MFU figures.
How to try it
The shortest route is the install command in the README. It requires Python 3.11 or newer:
pip install torch_tpu==0.2.0 --extra-index-url https://download.pytorch.org/whl/cpu
The --extra-index-url flag matters. Without it, the README warns, pip installs the CUDA build of torch. PyPI lists only Linux x86_64 wheels for 0.2.0 and no source distribution, so by that file list this command should fail on macOS and Windows. We did not run it on those systems.
For device setup, the developer blog uses torch.device("tpu"). For Hugging Face models, Google says Transformers v5 can deploy to TPUs with device_map="tpu". The tutorial notebooks must run from a TPU VM, and the README suggests a v6e GCE instance as the easiest option.
What to watch
Three things come next: whether the VeRL TPU backend and the TPU agent suite leave preview, whether version numbers and interfaces settle after 0.2.0, and whether anyone outside Google reproduces the MFU figures on other TPU generations. We have not found published pricing or regional availability.