AWS Trainium
Reduce costs while achieving great performance for training generative AI and deep learning.
AWS created the Trainium series of AI processors specifically for AI inference and training in order to provide great performance at a low cost.
Amazon Elastic Compute Cloud (Amazon EC2) Trn1 instances, which have training costs up to 50% lower than similar Amazon EC2 instances, are powered by the first-generation AWS Trainium processor. Numerous clients are recognising the cost and performance advantages of Trn1 instances, including Databricks, Ricoh, NinjaTech AI, and Arcee AI.
The performance of the AWS Trainium 2 chip is up to four times that of the original Trainium. The most potent EC2 instances for training and deploying models with hundreds of billions to trillions of parameters are Trainium2-based Amazon EC2 Trn2 instances, which are specifically designed for generative AI. Compared to the current generation of GPU-based EC2 P5e and P5en instances, Trn2 instances offer 30–40% better pricing performance. AWS own chip-to-chip interface, NeuronLink, connects 16 AWS Trainium 2 chips in Trn2 instances.
To create a wide range of next-generation generative AI applications, Trn2 instances can be used to train and implement the most complex models, such as diffusion transformers, multi-modal models, and large language models (LLMs). The largest models that need more memory and bandwidth than standalone EC2 instances can supply are best suited for Trn2 UltraServers, a brand-new EC2 service that is currently in preview.
New capabilities are made possible by the UltraServer design’s usage of NeuronLink to link 64 AWS Trainium 2 chips from four Trn2 instances into a single node. UltraServers contribute to industry-leading response times for inference, resulting in the greatest real-time experiences. Compared to isolated instances, UltraServers improve model training performance and efficiency by facilitating quicker collective communication for model parallelism.
Using native support for well-known machine learning (ML) frameworks like PyTorch and JAX, you can begin training and deploying models on Trn2 and Trn1 instances.
Advantages
Cost-effective, high-performing generative AI
For generative AI training and inference, Trn2 UltraServers and instances offer ground-breaking performance in Amazon EC2. With 64 AWS Trainium 2 chips connected via NeuronLink, our exclusive chip-to-chip interconnect, each Trn2 UltraServer can process 12.8 terabits per second (Tbps) of Elastic Fabric Adapter (EFA) networking, 6 TB of HBM3 with 185 terabytes per second (TBps) of memory bandwidth, and up to 83.2 petaflops of FP8 computation.
Each Trn2 instance can give up to 20.8 petaflops of FP8 computation, 1.5 TB of HBM3 with 46 TBps of memory bandwidth, and 3.2 Tbps of EFA networking thanks to its 16 AWS Trainium 2 processors connected to NeuronLink. With up to 16 Trainium chips, the Trn1 instance offers up to 1.6 Tbps of EFA networking, 512 GB of HBM with 9.8 TBps of memory bandwidth, and up to 3 petaflops of FP8 computation.
Support for machine learning frameworks and libraries natively
You can concentrate on creating and implementing models and speeding up your time to market by using the AWS Neurone SDK to extract the maximum performance from Trn2 and Trn1 instances. JAX, PyTorch, and necessary libraries like Hugging Face, PyTorch Lightning, and NeMo are all natively integrated with AWS Neurone. More than 100,000 models are supported by AWS Neurone on the Hugging Face model cluster, including well-known models like Stable Diffusion XL and Meta’s Llama family of models.
In addition to offering rich insights for profiling and debugging, it optimises models right out of the box for distributed training and inference. In addition to third-party services like Ray (Anyscale), Domino Data Lab, and Datadog, AWS Neurone interfaces with Amazon SageMaker, Amazon Elastic Kubernetes Service (Amazon EKS), Amazon Elastic Container Service (Amazon ECS), AWS ParallelCluster, and AWS Batch.
Cutting-edge AI optimisations
Trainium chips are optimised for FP32, TF32, BF16, FP16, and the new configurable FP8 (cFP8) data format in order to provide excellent performance while fulfilling accuracy objectives. AWS Trainium 2 features hardware optimisations for 4x sparsity (16:4), micro-scaling, stochastic rounding, and specialised collective engines to enable the rapid rate of invention in generative AI.
Created for research into AI
You can create novel model architectures and highly optimised compute kernels that surpass current methods by using the Neurone Kernel Interface (NKI), which provides direct access to instruction set architecture (ISA) through a Python-based environment with a Triton-like interface.
Designed with sustainability in mind
Compared to Trn1 instances, Trn2 instances are intended to be three times more energy efficient. Comparable accelerated computing EC2 instances are up to 25% less energy-efficient than Trn1 instances. When training ultra-large models, these examples assist you in achieving your sustainability objectives.
