In this blog, we will discuss NVIDIA HGX H200 price and comparison with HGX H100 in detail.
Introducing the HGX H200 from NVIDIA
NVIDIA unveiled its next top-tier AI chip, the HGX H200. It is a GPU designed especially for AI model deployment and training, which propels the growth in generative AI capabilities.
Based on the cutting-edge H200 Tensor Core GPU with increased memory capacity, the most recent NVIDIA HGX H200 can handle large data loads for generative AI and high-performance computing jobs. Additionally, the Hopper architecture from NVIDIA forms the foundation of the recently released HGX H200.
HGX H200 from NVIDIA is better than H100
How is the NVIDIA HGX H200 better than the H100?
The NVIDIA HGX H200 is essentially identical to the H100, except for memory.Its memory changes, however, represent a notable improvement. The new GPU is the first to use HBM3e, a faster memory specification. The HBM3e memory supports the chip’s “Inference” features, which use a trained model to generate predictions, text, or images. The NVIDIA HGX H200’s 1.1 TB HBM3e memory increases the GPU’s memory bandwidth to 4.8 TB per second, up from 3.35 TB per second in the H100. It has 141 GB of memory, up from 80 GB with the previous generation.
Compared to its predecessor, the NVIDIA A100, the capacity is almost doubled and the bandwidth is increased by 1.4 times. In addition, NVIDIA claims that the newest processor will outperform the HGX H100 by providing nearly twice the inference speed while running the Llama 2 large language model.
The advantages of upgrading the HBM memory in this new NVIDIA HGX H200 processor were actually highlighted by NVIDIA. “The addition of faster and larger HBM memory improves performance in computationally intensive tasks, such as generative AI models and high-performance computing applications,” said Ian Buck, NVIDIA’s Vice President of High-Performance Computing Products, in a video presentation that was made public on November 14. This improvement results in higher GPU efficiency and usage.
The NVIDIA HGX H200 Distribution
There is competition among government agencies, startups, and large organisations for a limited number of these NVIDIA HGX H200 chip.
Major cloud providers, including Google Cloud, Amazon Web providers, Oracle Cloud Infrastructure, and Microsoft Azure, have already committed to purchasing the newly released NVIDIA HGX H200 GPU, the company revealed. This GPU may be deployed in both four-way and eight-way configurations, guaranteeing compatibility with earlier HGX H100 software and hardware.
Server hardware partners as ASRock Rack, Asus, Dell, GIGABYTE, Hewlett Packard Enterprise, Lenovo, and others can upgrade their HGX H100 systems with the new NVIDIA HGX H200 processor.
On availability in the second quarter of 2024, the NVIDIA HGX H200 will compete with AMD’s MI300X GPU. Like the NVIDIA HGX H200, AMD’s device has more memory than its predecessors, which makes it possible to fit huge models on the hardware for effective inference processing.
Read more on NVIDIA HGX powered servers
NVIDIA HGX H200 Price
The upcoming chips are anticipated to have a high price tag when the NVIDIA HGX H200 is released. The previous HGX H100s are priced between $25,000 and $40,000 each, though NVIDIA has not revealed the precise prices, according to CNBC. The total cost can be high because thousands of these chips are frequently needed to operate at optimal performance. NVIDIA spokesperson Kristin Uchiyama reminds customers that partners decide prices.
The NVIDIA HGX H200: Effect on the HGX H100?
According to NVIDIA, the AI industry’s high demand for HGX H100 chips matches its claims. The efficiency with which these chips process the enormous volumes of data needed to run and train generative picture tools and large language models makes them highly respected. Because HGX H100s are so scarce, their value has increased to the point where businesses are using them as loan collateral. Owning HGX H100s has become a hot subject in Silicon Valley, and firms are working together to share access to these highly sought-after chips.
Uchiyama claims that the launch of the NVIDIA HGX H200 won’t affect the manufacture of the HGX H100 since NVIDIA intends to increase the total supply year-round and has a long-term commitment to obtaining more supply. The increased demand for generative AI implies that the market for these chips may continue to be strong even though NVIDIA is reportedly tripling the output of HGX H100 in 2024. This is especially true with the release of the even more sophisticated NVIDIA HGX H200.
