Google announced the open-source release of TorchTPU, a native PyTorch backend for its Tensor Processing Units (TPUs), on Tuesday. TorchTPU enables AI developers to run PyTorch models and operations directly on TPU devices, facilitating easier migration of existing PyTorch applications to Google Cloud with minimal code changes, according to techcrunch.com.
TorchTPU includes custom ATen kernels for PyTorch operations, a compilation cache to optimize execution of XLA computations, and support for distributed training methods such as data and tensor parallelism. The project also provides examples for various models including Llama, Qwen, ResNet, and minGPT, along with integration with torch.compile() for graph mode execution. Google has published a user guide and interactive notebooks to assist developers in adopting TorchTPU, as detailed on github.com.
The release positions Google alongside Amazon, which recently launched TorchNeuron to support its Trainium AI chips, in challenging Nvidia’s dominance in AI hardware. Nvidia’s GPUs have been the default for AI applications, with PyTorch—used by an estimated 85% of AI apps—originally optimized for Nvidia hardware. By enabling PyTorch compatibility with TPUs, Google aims to attract AI developers to its cloud platform and diversify the AI hardware ecosystem, techcrunch.com reports.
TorchTPU supports Python 3.11+ and is available now on GitHub, with detailed documentation at torchtpu.readthedocs.io. This release marks a key step in expanding hardware options for AI developers using PyTorch, the most widely used AI framework.