Amazon EC2 Trn1 Instances
AWS Trainium-powered Amazon Elastic Compute Cloud (EC2) Trn1 instances are designed specifically for high-performance deep learning (DL) training of gen AI models, such as latent diffusion models and large language models (LLMs). Compared to other similar Amazon EC2 instances, EC2 Trn1 instances can save up to 50% on the cost of training. Trn1 instances can be used to train generative AI and 100B+ parameter DL models for a wide range of applications, including fraud detection, recommendation, image and video generation, question answering, code generation, and text summarization.
Developers can deploy models on the AWS Inferentia chips and train models on AWS Trainium with the aid of the AWS Neuron SDK. Because of its natural integration with frameworks like PyTorch and TensorFlow, you can keep training models on EC2 Trn1 instances with your current code and workflows.
Benefits
Reduce training times for 100B+ parameter models
EC2 Trn1 instances shorten training durations from months to weeks or even days and are specifically designed for high-performance DL. You can create more creative models, iterate more quickly, and boost productivity with shorter training periods. For models that benefit from more network capacity, Trn1 instances provide a time-to-train that is up to 20% faster than Trn1 instances.
Lower your fine-tuning and pre-training costs
In comparison to other similar Amazon EC2 instances, Trn1 instances offer up to 50% cost-to-train savings while delivering outstanding performance.
Use your existing ML frameworks and libraries
To extract the complete performance of EC2 Trn1 instances, use the AWS Neuron SDK. With Neuron, you can train models on Trn1 instances using your current code and workflows while utilizing well-known machine learning frameworks like PyTorch and TensorFlow. Refer to the Neuron documentation’s popular model examples to get started with Trn1 instances right away.
Scale up to 6 exaflops with EC2 UltraClusters
Second-generation Elastic Fabric Adapter (EFAv2) network bandwidth of up to 800 Gbps is supported by EC2 Trn1 instances. For network-intensive models, Trn1 instances offer even better performance by supporting up to 1600 Gbps of EFAv2 network capacity. on order to expand up to 30,000 Trainium chips, which are joined by a nonblocking petabit-scale network to offer six exaflops of computing power, both instances are set up on EC2 UltraClusters.
Features
Up to 3 petaflops with AWS Trainium
Up to 16 AWS Trainium processors, designed specifically to speed up DL training and provide up to 3 petaflops of FP16/BF16 computation power, power Trn1 instances. There are two second-generation NeuronCores on each chip.
Up to 512 GB high-bandwidth accelerator memory
Each EC2 Trn1 instance includes 512 GB of shared accelerator memory (HBM) with a total memory bandwidth of 9.8 TB/s to facilitate effective data and model parallelism.
High-performance storage and networking
Each Trn1 instance provides up to 1600 Gbps of EFAv2 networking capacity to facilitate the training of network-intensive models, including Mixture of Experts (MoE) and Generative Pre-Trained Transformers (GPT). The maximum EFAv2 bandwidth supported by each EC2 Trn1 instance is 800 Gbps. By improving collective communications effectiveness by up to 50% over first-generation EFA, EFAv2 expedites distributed training. For quick workload access to big datasets, these instances additionally enable up to 8 TB of local NVMe solid state drive (SSD) storage and up to 80 Gbps of Amazon Elastic Block Store (EBS) connectivity.
NeuronLink interconnect
NeuronLink is a high-speed, nonblocking interconnect that Trn1 instances support up to 768 GB/s for quick networking across Trainium processors and efficient group communications.
State-of-the-art data types and DL optimizations
EC2 Trn1 instances are tuned for FP32, TF32, BF16, FP16, UINT8, and the new configurable FP8 (cFP8) data type in order to provide good performance while fulfilling accuracy objectives. Trn1 instances are flexible and expandable to train continuously changing DL models thanks to a number of advances that support the rapid pace of generative AI and DL innovation. Trn1 instances allow dynamic input shapes with software and hardware improvements. They support custom operators written in C++ to enable future support for more operators. In addition, they support stochastic rounding, which is a probabilistic rounding technique that outperforms legacy rounding options in terms of accuracy and performance.
