A thorough examination of TPU efficiency and lifecycle emissions in the design of sustainable AI
What is Compute Carbon Intensity (CCI) and Why It Matters?
Google is trying to lower the carbon intensity of AI systems by enhancing hardware efficiency, optimising software, and using carbon-free energy to power AI models as AI continues to open up new avenues for corporate expansion and societal advantages.
A first-of-its-kind study on the lifetime emissions of its Tensor Processing Unit (TPU) hardware is being released today. The carbon-efficiency of AI workloads has increased thrice as a result of more efficient TPU hardware design over the course of two generations, from TPU v4 to Trillium.
Its life-cycle assessment (LCA), which uses observational data from raw material extraction and production to energy usage during operation, offers the first comprehensive estimate of emissions from an AI accelerator. These results allow us to compare efficiency over generations and give us a glimpse of the average, chip-level carbon intensity of Google’s TPU technology.
Compute Carbon Intensity (CCI): An Introduction
In order to calculate the total life-cycle emissions of five TPU models and comprehend how hardware design choices have affected their carbon efficiency, its study looked at these models. It created a new metric called Compute Carbon Intensity (CCI) to evaluate emissions in relation to compute performance and allow for apples-to-apples comparisons between chips. This metric can promote better transparency and innovation in the industry.
The carbon emissions of an AI accelerator chip are quantified by Compute Carbon Intensity and are expressed in grammes of CO2e per Exa-FLOP per compute unit. For a particular AI workload, such as training an AI model, lower CCI scores indicate reduced emissions from the AI hardware platform. Google presents the findings of its efforts to improve the carbon efficiency of its TPUs, which tracked using CCI.
Important features
Google’s TPUs now have far higher carbon efficiency: Over a 4-year period, Google study discovered that its TPU chips’ Compute Carbon Intensity improved three times from TPU v4 to Trillium. By selecting more recent TPU generations, such as its 6th-generation Trillium TPU, its clients not only receive state-of-the-art performance but also produce lower carbon emissions for the same AI task.
Emissions from operational electricity are crucial: Nowadays, the great majority of a Google TPU’s lifetime emissions (more than 70%) come from operational electrical emissions. This emphasises how crucial it is to increase AI processors’ energy efficiency and lower the carbon intensity of the electricity that drives them. The primary goal of Google’s attempts to run on carbon-free energy (CFE) around-the-clock on all grids where it operates by 2030 is to lower operational electricity usage, which is the biggest source of TPU emissions.
Manufacturing is important: Even while operational emissions account for the majority of an AI chip’s lifetime emissions, manufacturing-related emissions are still significant and will rise in proportion to overall emissions as it use carbon-free energy to lower operational emissions. Google can focus its manufacturing decarbonisation efforts on the projects that will have the biggest impact according to the study’s thorough manufacturing life cycle assessment. By using more environmentally friendly production techniques and materials, it is actively collaborating with its supply chain partners to lower these emissions.
Rapid advances in AI model and algorithm creation are complemented by its substantial gains in AI hardware carbon-efficiency in this study. Beyond the scope of this study, ongoing AI model optimisation is lowering the number of calculations needed to achieve a particular model performance. A laptop can now run some models that formerly needed a supercomputer, and at Google, it is utilising methods like speculative decoding and Accurate Quantised Training to further boost model efficiency. Future research aims to quantify the effect of software design on carbon-efficiency, and it anticipate that model developments will continue to unlock carbon-efficiency improvements.
Collaborating toward Sustainable AI future
Google can focus its efforts to keep improving the carbon efficiency of its TPUs with the thorough approach it has taken here.
Though it’s only the first phase, this life-cycle analysis of AI hardware is a crucial first step in measuring and disseminating the carbon-efficiency of its AI systems. Google will keep looking into other facets of AI’s environmental footprint, like software efficiency improvements and emissions from AI models.
By working together, Google can maximizes AI’s revolutionary potential while reducing its negative environmental effects.
