First HBM3e Processor in the World Offers Revolutionary Memory and Bandwidth; Capability to Connect Multiple GPUs for Outstanding Performance; Design of Scalable Servers
The next-generation NVIDIA GH200 Grace Hopper platform, developed for the age of accelerated computing and generative AI, is now available. It is based on a new Grace Hopper Superchip and has the first HBM3e processor in the world.
The new platform, which spans massive language models, recommender systems, and vector databases, was developed to handle the most complicated generative AI workloads in the world. It will be offered in a variety of configurations.
The dual configuration consists of a single server with 144 Arm Neoverse cores, eight petaflops of AI performance, and 282GB of the most recent HBM3e memory technology. It provides up to 3.5x more memory capacity and 3x more bandwidth than the current generation offering.
According to Jensen Huang, founder and CEO of NVIDIA, “data centers require accelerated computing platforms with specialized needs to meet surging demand for generative AI.” “The new GH200 Grace Hopper Superchip platform delivers this with exceptional memory technology and bandwidth to improve throughput, the capability to connect GPUs to aggregate performance without compromise, and a server design that can be easily deployed across the entire data center.”
The new platform makes use of the Grace Hopper Superchip, which can be linked to other Superchips via NVIDIA NVLinkTM so they may cooperate in the deployment of the massive models required for generative AI. When used in dual mode, this fast, coherent technology grants the GPU complete access to the CPU memory, giving a combined 1.2TB of quick memory.
The new platform can run models that are 3.5 times larger than those of the previous generation while enhancing performance thanks to memory bandwidth that is 3 times quicker thanks to HBM3e memory, which is 50% faster than existing HBM3.
Increasing Interest in Grace Hopper
Leading suppliers already have products based on the Grace Hopper Superchip, which was previously unveiled. The next-generation Grace Hopper Superchip platform with HBM3e is fully compliant with the NVIDIA MGXTM server specification introduced at COMPUTEX earlier this year to promote widespread use of the technology. Any system maker can easily and affordably include Grace Hopper into more than 100 different server configurations using MGX.
Availability
In Q2 of the calendar year 2024, leading system makers are anticipated to deliver systems based on the platform.
Learn more about Grace Hopper by watching Huang’s keynote speech from SIGGRAPH on demand.
[…] NVIDIA GH200 Grace Hopper Superchip, which debuted on the MLPerf industry benchmarks, completed all data center inference tests while […]