Can InfiniBand Shake Ethernet?

Some people might think that with all the investment in artificial intelligence systems today, the boom in InfiniBand interconnect sales would cannibalize sales of high-end Ethernet interconnects in data centers. But this is not the case.

According to IDC’s latest market research, hyper-scaler enterprises, cloud builders, and some HPC centers and large enterprises already have enough 200 Gb/sec and 400 Gb/sec networks built out that the InfiniBand and Ethernet markets can grow simultaneously.

Of course, Ethernet is everywhere, from edge to campus to data center, unlike InfiniBand, which is specifically used in data centers. Therefore, it is important to differentiate between Ethernet switch sales in data centers and elsewhere. Based on statistics from the third quarter of 2023, IDC said sales of Ethernet switches entering data centers increased 7.2% year over year, a reasonable increase and reflecting the fact that as prices fall within the speed range of each port, throughput continues to rise.

IDC did not provide sales revenue figures for data center Ethernet switches in this public statement about its Ethernet tracker, but we have tracked IDC’s historical data, and our calculations show that data center Ethernet switch revenue reached $4.8 billion, unchanged from the previous quarter, Q2 2023, accounting for 41% of all Ethernet switch sales.

Worldwide Top5 Ethernet Switch Companies

IDC stopped providing port growth rates by type and speed back in Q4 2022, which means we can’t count ports being shipped into data centers, but we’ve made estimates to fill that gap. We openly acknowledge that these speculations are based on the interplay of volume and price declines as the supply chain loosens in the area of data center Ethernet switch. There is still a backlog, though, so prices won’t plummet. Regardless, our best guess is that 32.1 million ports will be shipped into data centers in Q3 2023, which would imply a 20.3% year-over-year increase in port shipments and a 24.5% sequential increase from Q2 2023.

That’s a far cry from Nvidia’s 5x growth in InfiniBand networking revenue in its most recent quarter. Over the past 12 months, we estimate in our model that Nvidia’s InfiniBand revenue grew 3.2x to $ 5.53 billion. (This includes switches, network interfaces, DPUs, cables, and software, not just switches.) But the data center Ethernet switch market still has an annualized operating rate of about $20 billion, if switching accounts for about half of InfiniBand’s revenue. Ethernet switching is still about 7 times larger than InfiniBand switching, and more and more AI clusters move to Ethernet, making Ethernet technology comparable to InfiniBand so that companies do not have to deploy anything in addition to Ethernet except in rare circumstances.

IDC said that in the non-data center part of the Ethernet switch market, sales grew faster as enterprises upgraded campus networks and built edges, growing 22.2% in the third quarter and 36.5% in the first three quarters of 2023. Calculating based on historical IDC data and growth trends, we see the non-data center portion of the market at $6.9 billion in sales, and we estimate nearly 250 million ports shipped at various speeds.

All told, the Ethernet switch market for data centers, campuses and edges reached $11.7 billion in the third quarter of 2023, growing 15.8% year over year. The companion Ethernet router market declined 9.4% to just under $3.7 billion, which is not surprising as more routers are being built using commodity chips that include switching and routing functions.

Router and Switch Revenue

As we’ve been doing for years, we tried to collate revenue and ports by Ethernet switch speed, which is much easier when IDC gives us a little more data than it currently provides in its Ethernet public statements. Obviously, this is no coincidence. We’ve been estimating port numbers since Q4 2022. According to our estimations, the situation in the third quarter of 2023 may be as follows:

Possible scenarios for Q3 2023

Possible pricing trend lines

That’s what’s interesting about all of this. In the data center field, sales of 200 Gb/sec and 400 Gb/sec Ethernet switches increased by 44% year-on-year, and port shipments increased by 63.9%. Sales of 100 Gb/sec Ethernet switches in data centers as well as edge network and campus increased 6%.

IDC also revealed another piece of news: Original design manufacturers (ODMs) accounted for 14.7% of data center sales, with sales of $705 million in the quarter, up 7.4% year-over-year from $656 million in the same period last year. In its comment on Ethernet switch vendors, IDC said that 72.2% of Cisco Systems’ Ethernet switch revenue comes from outside the data center, meaning only 27.8% comes from inside the data center (based on Cisco’s 45.1% Ethernet market share). We can calculate Cisco’s Ethernet switch sales in data center in the third quarter of 2023 to be $1.48 billion according to market data. Arista Networks is Cisco’s main competitor in the data center except ODM, with a 10.6% Ethernet switching market share and a 91.1% market share. Data centers accounted for $1.14 billion of its sales in the third quarter of 2023. In the third quarter of 2023, Cisco, Arista and ODM’s data center Ethernet switch sales were $3.32 billion, accounting for 69.2% of the market share.

By the way, while Hewlett Packard Enterprise has a fast-growing Ethernet switch business thanks to its acquisition of Aruba Networks, only 8% of its sales, or about $73 million, come from data centers, according to IDC. We think this will change over time, and we’d remind you that this number will also include sales of the Ethernet “Rosetta” Slingshot variant that HPE sells for HPC and AI clusters.

First, despite fierce competition, Cisco has done a pretty good job of protecting its data center switching business and expanding its business. Not only did it benefit from the wave of campus upgrades, but its data center business grew by 12.9%. We estimate that Arista’s data center switching business grew nearly three times as fast as the overall market during the quarter, or about 20%. ODM is growing at the pace of the market. If Cisco, Arista and ODM are excluded, the rest of the market would have seen a combined 5.4% decline in sales to $1.48 billion.

Second, as we’ve noted many times, the server market outside of AI servers is in decline, with Ethernet still being upgraded, even at a modest pace, and given the existing supply chain constraints, which shows how networking is rising in importance in data centers.