The recent launch of Intel\u2019s Gaudi-3, which utilizes RoCE for Scale-UP interconnection, along with Jim Keller\u2019s discussions on replacing NVLink with Ethernet, has brought attention to this innovative approach. Notably, Tenstorrent, where Jim Keller is involved, has cleverly implemented inter-chip network interconnection using Ethernet. Therefore, it\u2019s pertinent to address the challenges and requirements for Ethernet to replace NVLink.<\/p>\n\n\n\n
Replacing NVLink with Ethernet is not merely a matter of adopting a new transport protocol; it necessitates a series of modifications to the GPU architecture. Essentially, the issue is equivalent to figuring out how to hang High Bandwidth Memory (HBM) on an Ethernet network and achieve both Scale-Out and a series of communication optimizations to meet computational demands, such as In-Network Computing exemplified by SHARP. Globally, there are only a few individuals capable of addressing this multifaceted problem, and it\u2019s clear that UltraEthernet has not fully grasped the concept.<\/p>\n\n\n\n
To move forward, the following key questions must be addressed:<\/p>\n\n\n\n
\n
Latency Boundary: What is the acceptable latency boundary? The link latency caused by high-throughput, high-speed SerDes FEC, and interconnection beyond the scale of tens of thousands of cards is inevitable. These issues cannot be resolved simply by modifying a packet protocol or introducing HPC-Ethernet.<\/li>\n\n\n\n
Transmission Semantics: What are the semantics of the transmission? Network professionals typically understand basic SEND\/RECV operations. For instance, UEC\u2019s definition of Reliable Unordered Delivery for Idempotent operations (RUDI) is a technical misstep. While it satisfies the commutative and idempotent laws, it fails to address how certain operations, like the idempotence of reduction addition, can be implemented. Moreover, optimizations based on the associative law, which are unsupported for the fine-grained memory access on NVLink, are also necessary. More broadly, the semantics must evolve to that of a Semi-Lattice.<\/li>\n\n\n\n
Pooling Larger Memory on NVLink: How to address the pooling of larger memory on NVLink? This involves resolving the time\/space trade-offs for Compute Bound operators in computational problems, such as KV Cache.<\/li>\n\n\n\n
Dynamic Routing and Congestion Control: The ability to dynamically route and control congestion in a 1:1 non-converging Lossless network is not a significant issue for clusters with tens of thousands of cards through hardcoded tuning. However, for clusters at the scale of hundreds of thousands to millions of cards, which may even require RDMA for long-distance transmission, no commercial vendor has yet solved these problems.<\/li>\n<\/ul>\n\n\n\n