A single 1% packet-drop rate on a 1,024-GPU H100 cluster can burn roughly $250,000 per week in idle compute. That isn’t a theory. It’s why hyperscalers and HPC labs obsess over lossless, low-latency interconnects. InfiniBand RDMA is the technology that makes those interconnects possible.<\/p>\n\n\n\n
Remote Direct Memory Access (RDMA) lets one server read from or write to another server’s memory without waking either CPU or copying data through the operating system kernel. InfiniBand was built specifically for this. While Ethernet with RoCEv2 can run RDMA over a familiar stack, InfiniBand RDMA offers native losslessness and sub-microsecond latency that remains hard to match.<\/p>\n\n\n\n
In this article, we’ll walk through how InfiniBand RDMA actually works: verbs, queue pairs, memory regions, and the connection setup flow. Then we’ll connect it to AI training, GPUDirect RDMA, and the physical layer that often gets underestimated.<\/p>\n\n\n\n