NVIDIA DGX H100 Introduction

The NVIDIA DGX H100 system is a universal system designed for AI infrastructure and workloads. Here is a summary of the main hardware and component features of the system:

The GPU Board Tray is the key component assembly area of the entire AI server. The core part is the GPU Board Tray, which mainly contains GPU components, module boards, and NVSwitches.

The CPU Motherboard Tray is the core component of the server (including both ordinary servers and AI servers), which contains the CPU motherboard, system memory, network card, PCIE Switch, and other components.

• Graphics processor: 8 NVIDIA H100 GPUs, providing 640 GB of GPU memory.
• Central processor: 2 Intel Xeon 8480C PCIe Gen5 CPUs, each with 56 cores, clock speed ranging from 2.0 GHz to 3.8 GHz.

• NVLink: 4 fourth-generation NVLinks, providing 900 GB/s of GPU-to-GPU bandwidth.
• Storage:

Operating system storage: 2 1.92 TB NVMe M.2 SSDs (RAID 1 array).

Data cache storage: 8 3.84 TB NVMe U.2 SEDs (RAID 0 array).

• Network:

Cluster network: 4 OSFP ports, supporting InfiniBand (up to 400Gbps) and Ethernet (up to 400GbE).

Storage network: 2 NVIDIA ConnectX-7 dual-port Ethernet cards, supporting Ethernet (up to 400GbE) and InfiniBand (up to 400Gbps).

• System memory: Each 32 DIMMs provides 2 TB of memory.
• Out-of-band system management (BMC): Supports Redfish, IPMI, SNMP, KVM, and Web user interface.
• In-band system management: 3 dual-port 100GbE and 10GbE RJ45 interfaces. Mechanical & power specifications:

• Dimensions: 8U rack-mounted, height 14 inches, maximum width 19 inches, maximum depth 35.3 inches.
• Weight: Maximum 287.6 pounds (130.45 kg).
• Input voltage: 200-240 volts AC.
• Specifications for each power supply: Maximum 10.2 kilowatts, 3300 watts @ 200-240 volts, 16 amps, 50-60 hertz.
• Supports high-speed network connections, including InfiniBand and Ethernet, with speeds up to 400Gbps.