NVIDIA H100 GPU, a New Engine for Global AI Infrastructure, Delivers Dramatic Performance Gains
Santa Clara, California —Terms — March 22, 2022 —NVIDIA today announced the next generation of accelerated computing platforms with the NVIDIA Hopper™ architecture (https://www.nvidia.com/en-us/data-center/hopper-architecture/). Hopper delivers a performance breakthrough over previous generations, driving the next wave of AI data centers.
Named after Grace Hopper, a pioneer computer scientist in the United States, this new architecture inherits the NVIDIA Ampere architecture announced two years ago.
At the same time, NVIDIA announced the NVIDIA H100 (https://www.nvidia.com/en-us/data-center/h100), the first Hopper-based GPU with 80 billion transistors. The world’s largest and most powerful accelerator, the H100 is an NVIDIA NVLink(R) with innovative Transformer Engine, giant AI language model, deep recommender system, genomics, and high scalability to make evolve complex digital twins. as an interconnect.
NVIDIA Founder/CEO Jensen Huang said, “Data centers are becoming AI factories, processing and enhancing vast amounts of data to generate intelligence. The NVIDIA H100 is the global infrastructure for AI that companies use to accelerate their AI-driven business. It’s driving the framework.”
Technology Breakthrough H100
The NVIDIA H100 GPU sets new standards for accelerating AI and HPC at scale, enabling six game-changing innovations.
● World’s most advanced chip — Built with 80 billion transistors using state-of-the-art TSMC 4N processes, designed for NVIDIA’s accelerated computing needs. The H100 features significant advancements in AI, HPC, memory bandwidth, interconnect, and communications acceleration, including nearly 5 terabytes of external connections per second. The H100 is the first GPU to support PCIe Gen5 and also the first GPU to use HBM3, delivering 3TB/s of memory bandwidth. The 20 H100 GPUs can sustain the equivalent of global Internet traffic, allowing customers to offer advanced recommender systems and extended language models for real-time data inference.
● New Transformer Engine (https://blogs.nvidia.com/blog/2022/03/22/h100-transformer-engine/) — Currently, Transformer, the standard model choice for natural language processing, has been invented until now. It is one of the most important deep learning models ever created. The H100 Accelerator’s Transformer Engine is designed to make these networks six times faster than their predecessors, without compromising accuracy.
● Secure second-generation multi-instance GPU — MIG technology lets you split a single GPU into seven small, independent instances to handle a wide variety of tasks. The Hopper architecture extends the capabilities of MIG up to 7x compared to its predecessor by providing a secure multi-tenant configuration in the cloud environment for each GPU instance.
● Confidential Computing — The H100 is the world’s first accelerator with Confidential Computing capabilities that protect AI models and customer data during processing. Customers are not only for shared cloud infrastructure, but also for privacy-sensitive industries such as healthcare and financial services, Federated Learning (https://blogs.nvidia.co.jp/2021/ 12/) It is also possible to apply computer confidential information to 02/federated-learning-ai-nvidia-flare/).
● 4th Gen NVIDIA NVLink — To accelerate larger AI models, NVLink combined with a new external NVLink switch to extend NVLink as a scalable network beyond the server before using NVIDIA HDR Quantum InfiniBand. Connect up to 256 H100 GPUs with 9x the bandwidth of the generation.
● DPX Instructions (https://blogs.nvidia.com/blog/2022/03/22/nvidia-hopper-accelerates-dynamic-programming-using-dpx-instructions/) — New DPX instructions are routing optimization and genomics. Dynamic programming used in a wide range of algorithms, including, is up to 40 times faster than CPUs and up to 7 times faster than previous generation GPUs. This includes the Floyd-Warshall algorithm for finding the optimal route for autonomous robots in a dynamic warehouse environment, and the Smith-Waterman algorithm used in DNA and protein classification and sequence alignment folding. ..
The H100’s technological innovations extend NVIDIA’s leadership in AI inference and training, enabling real-time, immersive applications using giant AI models. The H100 uses the world’s most powerful monolithic Transformer language model, the Megatron 530B (https://nvidianews.nvidia.com/news/nvidia-brings-large-language-ai-models-to-enterprises-worldwide). to meet the sub-second latency required for real-time interactive AI, while delivering up to 30x the throughput of previous generation GPUs. The H100 enables researchers and developers to train large models such as the Mixture of Experts up to 9x faster with 395 billion parameters, reducing training time from weeks to days.
Wide adoption of NVIDIA H100
The NVIDIA H100 can be deployed in any type of data center, including on-premises, cloud, hybrid cloud, and edge. It will be available directly from leading cloud service providers, PC manufacturers and NVIDIA globally later this year.
NVIDIA’s 4th generation DGX™ system, the DGX H100 (https://nvidianews.nvidia.com/news/nvidia-announces-dgx-h100-systems-worlds-most-advanced-enterprise-ai-infrastructure) Equipped with eight H100 GPUs, it delivers 32 petaflops of AI performance with new FP8 precision. It enables large-scale language models, recommender systems, healthcare research, and scales that meet the large-scale computational demands of climate science.
All GPUs in the DGX H100 system are connected by 4th generation NVLink, which has a bandwidth of 900 GB/s, which is 1.5 times that of the previous generation. NVSwitch™ also connects all eight H100 GPUs via NVLink. The external NVLink switch connects up to 32 DGX H100 nodes to the network on the next-generation NVIDIA DGX SuperPOD™ supercomputer.
Hopper is widely supported by industry and is a leading cloud service provider, Alibaba Cloud, Amazon Web Services, Baidu AI Cloud, Microsoft Azure, Oracle Cloud (https://blogs.oracle.com/cloud-infrastructure/ post/). oracles-next-generation-ai-ml-services-nvidia-pushing-boundaries-of-innovation), TencentCloud and others will provide H100-based instances.
Additionally, leading global system makers Atos, BOXX Technologies, Cisco, Dell Technologies (https://www.dell.com/en-us/blog/making-it-easier-than-ever-for-ai-anywhere )/), Fujitsu, GIGABYTE (https://www.gigabyte.com/Press/News/1977), H3C, Hewlett Packard Enterprise (https://www.hpe.com/us/en/newsroom/press-release /2022/03/hpe-greenlake-edge-to-cloud-platform-delivers-greater-choice-and-simplicity-with-unified-experience-new-cloud-services-and-expanded-partner-ecosystem.html), Inspur, Lenovo, Nettrix, Supermicro will offer a variety of servers with H100 accelerators.
NVIDIA H100 of all sizes
The H100 is available in SXM and PCIe form factors to support a wide range of server design requirements. H100 GPU with NVIDIA ConnectX(R)-7 400Gb/s InfiniBand (https://www.nvidia.com/en-us/networking/infiniband-adapters/) and Ethernet (https://www.nvidia.com/en ) -us / networking / ethernet-adapters /) Accelerators converged in combination with SmartNIC will also be available.
NVIDIA’s H100 SXM will be available on HGX™ H100 server boards in 4-lane and 8-lane configurations for enterprises extending their applications to multiple GPUs within or across multiple servers. HGX H100-based servers deliver the best application performance for AI training and inference, as well as data analytics and HPC applications.
The H100 PCIe includes NVLink, which connects two GPUs with over seven times the bandwidth of PCIe 5.0, delivering exceptional performance for applications running on leading enterprise servers. Its form factor facilitates integration into existing data center infrastructure.
The new converged accelerator, the H100 CNX (https://www.nvidia.com/en-us/data-center/h100cnx), combines the H100 with the ConnectX-7 SmartNIC for multi-node AI training in enterprise data centers. It delivers breakthrough performance for I/O-intensive applications such as edge and edge 5G signal processing.
GPUs based on the NVIDIA Hopper architecture are ultra-fast NVLink-C2C interconnects (https://nvidianews.nvidia.com/news/nvidia) that communicate between CPU and GPU more than 7 times faster than PCIe 5.0. open-nvlink-for-custom-silicon-integration (https://nvidianews.nvidia.com/news/nvidia-announces-cpu-for-giant-ai-and-high-performance) It can also be combined with -computing -workloads). This combination (Grace Hopper Superchip) (https://nvidianews.nvidia.com/news/nvidia-introduces-grace-cpu-superchip) is an embedded module designed for large-scale HPC and AI applications.
NVIDIA software support
The NVIDIA H100 GPU is supported by powerful software tools that enable developers and businesses to build and accelerate applications ranging from AI to HPC. This includes NVIDIA AI for workloads such as voice, recommender systems, and hyperscale inference (https://nvidianews.nvidia.com/news/nvidia-ai-delivers-major-advances-in-speech- recommender-system-and-hyperscale -inference) Contains major software suite updates.
NVIDIA is also delivering over 60 updates (https://nvidianews) to its CUDA-X™ Collection, which includes libraries, tools, and technologies for accelerating workloads in quantum computing, 6G research , cybersecurity, genomics and drug discovery. .nvidia.com/news/nvidia-introduces-60+-updates-to-cuda-x-libraries-opening-new-science-and-industries-to-accelerated-computing) has been released.
Planned to be supplied
The NVIDIA H100 will be available in the third quarter.
For more information on NVIDIA Hopper and the H100, check out Jensen Huang’s GTC 2022 Keynote Replay. Register for GTC 2022 (https://www.nvidia.com/ja-jp/gtc/?ncid=ref-pr-242463) for free and attend sessions hosted by NVIDIA and industry leaders.
The invention of the GPU by (NVIDIA https://www.nvidia.com/ja-jp/) (NASDAQ display: NVDA) in 1999 spurred the growth of the PC gaming market explosively, modern computer graphics, high-performance computing, and artificial intelligence (AI) redefined. NVIDIA’s pioneering efforts in accelerated computing and artificial intelligence are replenishing trillions of dollars in industries such as transportation, healthcare, and manufacturing, and accelerating expansion in many other industries.
For more information, click this link: https://nvidianews.nvidia.com/
To the company press release details (2022/03/23-10:47)