Nvidia’s flagship AI chip reportedly 4.5x faster than the previous champion




Nvidia H100 Tensor Core GPU

Enlarge / A press image of the Nvidia H100 Tensor Core GPU.

Nvidia announced yesterday that its upcoming H100 “Hopper” Tensor Core GPU set new performance records during its debut in industry-standard MLPerf benchmarks, delivering results up to 4.5 times faster than the A100, currently Nvidia’s fastest production AI chip .

The MPerf benchmarks (technically called “MLPerfTM Inference 2.1”) measure “inference” workloads, which show how well a chip can apply a previously trained machine learning model to new data. A group of industry companies known as MLCommons developed the MLPerf benchmarks in 2018 to provide a standardized metric for communicating machine learning performance to potential customers.

Nvidia's H100 benchmark results match the A100 in fancy bar graph form.

Enlarge / Nvidia’s H100 benchmark results match the A100 in fancy bar graph form.


In particular, the H100 performed well in the BERT-Large benchmark, which measures natural language processing performance using the BERT model developed by Google. Nvidia credits this particular result to the Hopper architecture’s Transformer Engine, which specifically accelerates training transformer models. This means that the H100 can accelerate future natural language models similar to OpenAI’s GPT-3, which can compose written works in many different styles and hold conversational chats.

Nvidia is positioning the H100 as an advanced data center GPU chip designed for AI and supercomputing applications such as image recognition, large language models, image synthesis and more. Analysts expect it to replace the A100 as Nvidia’s flagship data center GPU, but it is still in development. US government restrictions imposed last week on the export of chips to China led to fears that Nvidia might not be able to deliver the H100 by the end of 2022, as part of its development takes place there.

Nvidia clarified in another SEC filing last week that the US government will allow continued development of the H100 in China, so the project appears to be back on track for now. According to Nvidia, the H100 will be available “later this year.” If the success of the previous generation A100 chip is any indication, the H100 could power a wide range of cutting-edge AI applications in the coming years.



Leave a Reply

Your email address will not be published. Required fields are marked *