Nvidia Hopper H100: Huge 4nm AI GPU


Nvidia took advantage of GTC to introduce Hopper, its new GPU architecture designed for the professional sector looking for hardware acceleration. All this is embodied in the H100, a huge GPU with 80 billion transistors, made in 4nm process technology.

For the mainstream market, Nvidia is first and foremost the leading chipmaker for video cards, and the GeForce series has dominated the market for more than a decade. But under the leadership of its leader Jensen Huang, the company has always looked far beyond the gamer market and very quickly positioned itself as a major player in the field of hardware acceleration. An area where demand is growing exponentially even as the resource requirements are huge in subjects such as artificial intelligence and model management (climate, road, etc.) among others.

For this particular sector, Nvidia has historically always had two different approaches. The first was to adapt its graphic architecture in two flavors, one for the general public and one for professionals – as in the case of Ampère -; the second sought to create two different architectures, each aimed at a specific market, as in the case of the Volta, which was specifically designed for the acceleration field.

Hopper supports this second approach. The architecture was designed for the acceleration domain to meet the expectations of the AI ​​or even the omniverse. And the least we can say is that two years after the GA100 chip (Ampere architecture), Nvidia is offering a pretty impressive H100 chip on paper. Consisting of 80 billion transistors spread over an area of ​​814 mm², it stands out quite clearly from its predecessor, which was “limited” to 54.2 billion transistors in an area of ​​828 mm². Numbers that are not misleading, Nvidia has ditched the 7nm engraving in favor of the 4nm offered by TSMC (N4 node). The chip also draws a maximum of 700W, which is much more than the previous generation’s 500W maximum.

Nvidia Hopper

With a built-in PCIe 5.0 interface, the chip is surrounded by a maximum of 80GB of dedicated HBM3 memory, enough to provide 3TB/s of bandwidth. The specific compute blocks Nvidia calls accelerators have been revisited, with the fourth-generation Tensor Core targeting AI in particular and claiming to be six times faster than the GA100 chip. The number of compute units like CUDA Cores is growing rapidly, growing from 6912 to 16896. This gives a raw performance three times higher than on the old generation accelerator, and this, it should be remembered, in just two years.

Nvidia has also introduced a new acceleration engine called the Transformer Engine. This is intended to speed up the processing of AI-related models with regards to real-time translation, query interpretation, image analysis, or even health and climate. Neural training that used to take days can now be completed in just a few hours. A feat that will interest his world, in particular Google, whose BERT algorithm uses this type of mechanism to better understand user queries and answer those questions more accurately. For example, Nvidia indicated that a job that used to take 7 days on 8,000 GPUs will now take just 20 hours with Hopper chips.

Nvidia Hopper

This new GPU will be offered to Nvidia partners starting in the third quarter. It can be purchased separately (PCIe format), as well as DGX racks that combine 8 modules, or even SuperPOD cabinets that have 32 modules. A maximum of 256 modules can be interconnected using an NVLink switch capable of linking at 70.4 TB/s between modules. Finally, supercomputers are already included in the program, in particular Nvidia’s Eos division – a supercomputer that the company will use itself and which it will offer to its partners – which will include 576 DGX racks or 4608 GPUs. Eos will offer 275 petaflops of processing power in FP64, making it the world’s second supercomputer after Fugaku (442 petaflops). Now it remains to wait for announcements from Nvidia in the field of the general public: most likely, the firm should announce the succession of Ampere in the coming months.

Leave a Comment