Google Joins Amazon to Challenge Nvidia’s AI Developer Dominance
The company has released TorchTPU, a software development kit that allows running PyTorch AI workloads natively on its proprietary TPUs without any need for a full-scale rewrite.
The effort comes one day after Amazon announced its similar TorchNeuron software, and seeks to tackle the issue of hardware lock-in within the AI software stack. Hyperscalers hope that by simplifying code portability, they will be able to weaken Nvidia’s control over the infrastructure for AI.
TorchTPU is developed in cooperation with the PyTorch team at Meta and brings TPUs to the list of native backend targets, together with GPUs and CPUs. PyTorch powers about 85% of current production AI applications where software dependencies kept developers locked into Nvidia’s CUDA-based programming stack.
TorchTPU becomes a successor to previously used compiled execution layers such as PyTorch/XLA and makes it possible to run, debug, and scale the models across cloud TPUs via configuration tweaks only. From an operational standpoint, the software release works well with ecosystem partners such as Hugging Face, vLLM, and SGLang, making sure that the framework remains compatible with many other frameworks.
In terms of reducing migration costs from a technical perspective, Google Cloud offers an economic solution for training and inference for enterprise clients, making sure that the cloud hyperscaler will be able to take advantage of its custom silicon investments. The joint effort of Google and Amazon marks a new trend in the battle against Nvidia, a strategy of building software ecosystems rather than competing at the level of hardware innovations.
As the cloud hyperscalers continue spending billions on custom AI chips, breaking CUDA’s software lock-in becomes an essential step in diverting market share away from Nvidia.
The recent development by Google of Torch TPU means that the company is setting a path to making it easy for enterprises to deploy across different hardware, indicating an even more competitive environment as the big cloud service providers continue to contest for market dominance against the monopoly of proprietary hardware in the AI world.
By reducing the technical friction involved in moving high-throughput models to custom chips, the cloud hyperscalers are working on reducing the total cost of ownership.
