Intel Shows Off New AI Processor, Named Gaudi 3
Photo: Doc. Intel
- Intel showed off its newest AI accelerator called Gaudi 3, which is claimed to have 50% better performance than the Nvidia H100.
Gaudi 3 was exhibited at the Vision 2024 conference held in Phoenix, Arizona, United States, after previously being revealed at the Intel AI Everywhere event in December 2023.
At that time, Intel CEO Pat Gelsinger predicted that Gaudi 3 would be a tough rival for Nvidia and AMD's AI chips. Gaudi 3 is the successor to Gaudi 2, made of two TSMC 5nm chips and equipped with 64 5th generation Tensor cores.
The HBM2e memory capacity reaches 128GB with a speed of 3.7Gbps and a bandwidth of 3.7 TB per second, as quoted by detikINET from Techspot, Thursday (11/4/2024).
The improvement compared to Gaudi 2 is quite significant, because the previous generation chip only has 24 Tensor cores, 96GB HBM2e memory with a speed of 3.27Gbps, and bandwidth of 2.45TB per second. Gaudi 3 is also available in 96GB SRAM variant and 12.8TB bandwidth per second.
Other important specifications are 128GB VRAM capacity, computing capability of 1,835 TFLOPS FP8 and BF16 matrix, and power consumption of 900 watts. For comparison, Gaudi 2 has 96GB VRAM, and has 835 TFLOPS in FP8 Matrix, 432 TFLOPS in BF16 Matrix, and power consumption is only 600W.
In addition, Gaudi 3 is also equipped with 24 200Gb Ethernet ports to make it more flexible and supports open networking standards. There is also PCIe support for various needs, including retrieval-augmented generation (RAG). According to Intel, Gaudi 3 has 4x faster AI computing capabilities in BF16, 1.5x higher memory bandwidth, and 2x higher network bandwidth than its predecessor, suitable for "training" generative AI (GenAI).
Meanwhile, when compared to the Nvidia H100, Intel claims that its newest AI chip is 50% faster when used for Llama2 7B and 13B computing, as well as for the GPT-3 175B parameter model. Apart from being faster, power consumption is also 40% more efficient in the same parameters.
Currently, Gaudi 3 chip samples have begun to be shipped to Intel partners, and mass production is scheduled to begin in the second half of this year. The air-cooled Gaudi 3 will start shipping in Q3 2024, while the new water-cooled variant will be available in Q4 2024.
0 Comments