Outstanding AI Performance with Habana Gaudi2 and Intel Xeon Scalable Processors

Intel has recently achieved remarkable training results in the MLCommons industry AI performance benchmark, MLPerf Training 3.0. Both the Habana® Gaudi®2 deep learning accelerator and the 4th Gen Intel Xeon Scalable processor delivered outstanding performance, highlighting Intel’s prowess in AI training.

The MLPerf results validate the total cost of ownership (TCO) value that Intel Xeon processors and Intel Gaudi deep learning accelerators offer to customers in the field of AI. Xeon processors with built-in accelerators are ideal for running high-volume AI workloads on general-purpose processors, while Gaudi delivers competitive performance for large language models (LLMs) and generative AI. Intel’s scalable systems, combined with optimized and user-friendly open software, lower the barriers for customers and partners to deploy a wide range of AI-based solutions across the data center, from the cloud to the intelligent edge.

The prevailing industry narrative suggests that generative AI and large language models can only run on Nvidia GPUs. However, the latest MLPerf Training 3.0 results demonstrate that Intel’s AI solutions provide compelling alternatives for customers who seek to break free from closed ecosystems that limit efficiency and scalability.

The MLPerf results showcase the performance of Intel’s products across various deep learning models. The maturity of Gaudi2-based software and systems for training was demonstrated at scale on the GPT-3 large language model, solidifying Gaudi2 as one of only two semiconductor solutions to submit performance results to the benchmark for LLM training of GPT-3.

Additionally, Gaudi2 offers substantial cost advantages to customers in terms of both server and system costs. The MLPerf-validated performance of Gaudi2 on GPT-3, computer vision, and natural language processing models, along with upcoming software advancements, positions Gaudi2 as an extremely compelling price/performance alternative to Nvidia’s H100.

On the CPU front, the 4th Gen Xeon processors with Intel AI engines showcased exceptional deep learning training performance. Customers can build single universal AI systems for data pre-processing, model training, and deployment using Xeon-based servers. This approach ensures the right balance of AI performance, efficiency, accuracy, and scalability.

The MLPerf results for the Habana Gaudi2 highlight its outstanding performance and efficient scalability, particularly in training generative AI and large language models. The results demonstrate Gaudi2’s impressive time-to-train on the 175 billion parameter GPT-3 model, as well as its near-linear scaling from 256 to 384 accelerators. Gaudi2 also delivered excellent training results on computer vision and natural language processing models, with performance increases compared to the previous submission, showcasing the growing software maturity of Gaudi2.

The software support for the Gaudi platform continues to mature, keeping pace with the increasing demand for generative AI and LLMs. Gaudi2’s GPT-3 submission was based on PyTorch and employed the popular DeepSpeed optimization library, enabling support for 3D parallelism and further optimizing scaling performance efficiency on LLMs. Future software releases in 2023’s third quarter are expected to bring significant performance improvements to Gaudi2.

The MLPerf results for the 4th Gen Xeon processors demonstrate that Intel Xeon processors provide enterprises with the out-of-the-box capabilities to deploy AI on general-purpose systems, eliminating the need for dedicated AI systems and associated complexities and costs. The results showcase the training efficiency of Xeon processors on various models, including BERT, ResNet-50, and RetinaNet.

MLPerf is considered the most reputable benchmark for AI performance, enabling fair and repeatable performance comparisons across different solutions. Intel’s commitment to transparency is evident as they have surpassed the 100-submission milestone and remain the only vendor to submit public CPU results with industry-standard deep-learning ecosystem software.

Furthermore, the results highlight the excellent scaling efficiency achieved using cost-effective and readily available Intel Ethernet 800 Series network adapters, which leverage the open-source Intel Ethernet Fabric Suite Software based on Intel oneAPI.

In summary, the MLCommons MLPerf Training 3.0 results underscore Intel’s impressive AI advancements, solidifying their competitive position in the field. With their portfolio of AI solutions, Intel provides customers with compelling alternatives to closed ecosystems, ensuring efficiency, scalability, and cost-effectiveness in AI training.