Trillium TPUs
The way Google cloud engage with technology is changing due to generative AI, which is also creating a great deal of efficiency opportunities for corporate effect. However, in order to train and optimise the most powerful models and present them interactively to a worldwide user base, these advancements need on ever-increasing amounts of compute, memory, and communication. Tensor Processing Units, or TPUs, are unique AI-specific hardware that Google has been creating for more than ten years in an effort to push the boundaries of efficiency and scale.
Many of the advancements Google cloud introduced today at Google I/O, including new models like Gemma 2, Imagen 3, and Gemini 1.5 Flash, which are all trained on TPUs, were made possible by this technology. Google cloud thrilled to introduce Trillium, Google’s sixth-generation TPU, the most powerful and energy-efficient TPU to date, to offer the next frontier of models and empower you to do the same.
Comparing Trillium TPUs to TPU v5e, a remarkable 4.7X boost in peak computation performance per chip is achieved. Google cloud increased both the Interchip Interconnect (ICI) bandwidth over TPU v5e and the capacity and bandwidth of High Bandwidth Memory (HBM). Third-generation SparseCore, a dedicated accelerator for handling ultra-large embeddings frequently found in advanced ranking and recommendation workloads, is another feature that Trillium has. Trillium TPUs provide faster training of the upcoming generation of foundation models, as well as decreased latency and cost for those models. Crucially, Trillium TPUs are more than 67% more energy-efficient than TPU v5e, making them Google’s most sustainable TPU generation to date.
Up to 256 TPUs can be accommodated by Trillium in a single high-bandwidth, low-latency pod. In addition to this pod-level scalability, Trillium TPUs can grow to hundreds of pods using multislice technology and Titanium Intelligence Processing Units (IPUs). This would allow a building-scale supercomputer with tens of thousands of chips connected by a multi-petabit-per-second datacenter network.
The next stage of Trillium-powered AI innovation
Google realised over ten years ago that a novel microprocessor was necessary for machine learning. They started developing the first purpose-built AI accelerator in history, TPU v1, in 2013. In 2017, Google cloud released the first Cloud TPU. Many of Google’s best-known services, including interactive language translation, photo object recognition, and real-time voice search, would not be feasible without TPUs, nor would cutting-edge foundation models like Gemma, Imagen, and Gemini. Actually, Google Research’s foundational work on Transformers the algorithmic foundations of contemporary generative AI Fwas made possible by the size and effectiveness of TPUs.
Compute performance per Trillium chip increased by 4.7 times
Since TPUs Google cloud created specifically for neural networks, Google cloud constantly trying to speed up AI workloads’ training and serving times. In comparison to TPU v5e, Trillium performs 4.7X peak computing per chip. We’ve boosted the clock speed and enlarged the size of matrix multiply units (MXUs) to get this level of performance. Additionally, by purposefully offloading random and fine-grained access from TensorCores, SparseCores accelerate workloads that involve a lot of embedding.
The capacity and bandwidth of High Bandwidth Memory (HBM) with 2X ICI
Trillium may operate with larger models with more weights and larger key-value caches by doubling the HBM capacity and bandwidth. Higher memory bandwidth, enhanced power efficiency, and a flexible channel architecture are made possible by next-generation HBM, which also boosts memory throughput. For big models, this reduces serving latency and training time. This equates to twice the model weights and key-value caches, allowing for faster access and greater computational capability to expedite machine learning tasks. Training and inference tasks may grow to tens of thousands of chips with double the ICI bandwidth thanks to a clever mix of 256 chips per pod specialised optical ICI interconnects and hundreds of pods in a cluster via Google Jupiter Networking.
The AI models of the future will run on trillium
The next generation of AI models and agents will be powered by trillium TPUs, and they are excited to assist Google’s customers take use of these cutting-edge features. For instance, the goal of autonomous car startup Essential AI is to strengthen the bond between people and computers, and the company anticipates utilising Trillium to completely transform the way organisations function. Deloitte, the Google Cloud Partner of the Year for AI, will offer Trillium to transform businesses with generative AI.
Nuro is committed to improving everyday life through robotics by training their models with Cloud TPUs. Deep Genomics is using AI to power the future of drug discovery and is excited about how their next foundational model, powered by Trillium, will change the lives of patients. With support for long-context, multimodal model training and serving on Trillium TPUs, Google Deep Mind will also be able to train and serve upcoming generations of Gemini models more quickly, effectively, and with minimal latency.
AI-powered trillium Hypercomputer
The AI Hypercomputer from Google Cloud, a revolutionary supercomputing architecture created especially for state-of-the-art AI applications, includes Trillium TPUs. Open-source software frameworks, flexible consumption patterns, and performance-optimized infrastructure including Trillium TPUs are all integrated within it. Developers are empowered by Google’s dedication to open-source libraries like as JAX, PyTorch/XLA, and Keras 3. Declarative model descriptions created for any prior generation of TPUs can be directly mapped to the new hardware and network capabilities of Trillium TPUs thanks to support for JAX and XLA. Additionally, Hugging Face and they have teamed up on Optimum-TPU to streamline model serving and training.
Since 2017, SADA (An Insight Company) has won Partner of the Year annually and provides Google Cloud Services to optimise effect.
The variable consumption models needed for AI/ML workloads are also provided by AI Hypercomputer. Dynamic Workload Scheduler (DWS) helps customers optimise their spend by simplifying the access to AI/ML resources. By scheduling all the accelerators concurrently, independent of your entry point Vertex AI Training, Google Kubernetes Engine (GKE), or Google Cloud Compute Engine flex start mode can enhance the experience of bursty workloads like training, fine-tuning, or batch operations.
Lightricks is thrilled to recoup value from the AI Hypercomputer’s increased efficiency and performance.
