With the growing adoption of artificial intelligence over the past 12 months, data center networking infrastructure is being pushed to its limits, with enterprises and cloud operators increasing capacity and looking at wholesale infrastructure improvements to meet demand.
Each segment of the data center operation is expecting an increase in spending, including the data center switch market, which a recent report by the Dell'Oro Group forecasts will expand by 50% this year.
While most of the growth in the past in the switch market has come from front-end networks, Sameh Boujelbene of the Dell'Oro Group said in a release: “This rapid growth [of AI] necessitates the deployment of thousands or even hundreds of thousands of accelerated nodes. Connecting these accelerated nodes in large clusters requires a data center-scale fabric, known as the AI back-end network, which differs from the traditional front-end network used mostly to connect general-purpose servers.”
This may force data center operators to rewire parts of the data centers to be fully optimized for cloud workloads, potentially increasing the costs of running the data center.
A key decision in the data center switch market is Ethernet or InfiniBand, the two most common networking communication standards in data centers. InfiniBand is increasingly being looked at by data center operators due to its very high throughput, low latency, and scalability, all of which make it appealing for high-performance computing (HPC) and AI.
NVIDIA, which manufactures InfiniBand bus adapters and network switches, has been promoting the technical proficiency of the standard for a while now, but some enterprises are still reluctant to adopt it. Ethernet has been the standard for generations, and while InfiniBand may sport better performance in some areas, it is also more costly, complex, and has interoperability problems with infrastructure.
Ethernet is also evolving to meet the demands of AI, with the formation of the Ultra Ethernet Consortium (UEC) hosted by The Linux Foundation with founding members AMD, Broadcom, Cisco, Intel, and others. The UEC aims to improve Ethernet stack architecture for AI and HPC by reducing tail latency through the adoption of packet spraying, in which every communication exchange is simultaneously used. Other upgrades are being investigated by the consortium.
According to the Dell’Oro Group, Ethernet is forecast to make the most revenue-share points in the next three years. “One could argue that Ethernet is one speed generation ahead of InfiniBand. Network speed, however, is not the only factor. Congestion control and adaptive routing mechanisms are also important,” said Boujelbene in the release.
Ethernet will also see a higher percentage of market share with Tier 2 and Tier 3 cloud service providers (CSPs), which are forecast to generate over $10 billion in annualized revenues by 2029. While Tier 1 CSPs will service most of the AI demand, lower-tier CSPs are expected to start offering AI-centric enterprise packages to meet growing local demand.
AI networks will also see improvements in networking speeds as more data centers adopt the 800 Gb/s standard. By 2025, this is expected to be the standard in most back-end network ports, two years after the first ports were commercially rolled out.
Related articles:
Bob Friday, Chief AI Officer for Juniper Networks, explains how the advanced technology is transforming operations.
Contact center leaders from 8x8, Awaken Intelligence, and 360insight discuss the importance of agent experience.
API security is going to be a foundational security capability that every organization will need to employ if they're going to leverage AI in any capacity.
