Top 5 Infrastructure for AI Articles in 2024

The mainstream use of artificial intelligence is causing a disruption in enterprises across all industries. As a result, most enterprises are keeping some or all of their AI model training and inferencing on-premises for a variety of reasons. What many find is that their existing infrastructure is not adequate and cannot support these new workloads.

While much attention has focused on meeting the compute demands of AI, there are comparable (and equally hard to address) issues with networking and storage.

2024 Infrastructure for AI Challenges and Solutions

Numerous Network Computing articles in 2024 focused on meeting the infrastructure demands of AI. Below is a list of our top 5 articles for the year, with a brief summary of each article.

1) Ethernet Holds Its Own in Demanding AI Compute Environments

AI workloads place new demands on the network elements in a data center. Ethernet remains a viable option as it is proving to be a robust and cost-effective solution for handling the demanding networking requirements of AI workloads. While alternatives like InfiniBand offer high performance, they come with challenges such as increased costs and complexity.

Ethernet, with its widespread adoption and simplicity, is being actively enhanced to meet AI demands through initiatives like the Ultra Ethernet Consortium, led by AMD, Cisco, Intel, and Microsoft. This consortium is focused on optimizing Ethernet's performance for low-latency, high-bandwidth tasks by addressing critical issues like tail latency and improving packet flow efficiency. For IT leaders, this means they can rely on existing Ethernet infrastructure while keeping costs and complexity under control, enabling scalable support for their growing AI efforts.

2) Cisco Report: Enterprises Ill-prepared to Realize AI’s Potential

The recent Cisco 2024 AI Readiness report found that only 13% of organizations are fully prepared to harness AI's potential. Most enterprises have significant gaps in infrastructure, skills, and data quality. Notably, 79% of companies lack sufficient GPUs to meet current and future AI demands, and 24% report inadequate AI-related expertise within their workforce. Additionally, 80% face challenges in data preprocessing and cleaning, which is critical for effective AI implementation. Despite these hurdles, 98% of organizations acknowledge an increased urgency to adopt AI technologies, with 85% aiming to demonstrate AI's business impact within 18 months. For IT managers in large enterprises, this underscores the necessity of investing in AI infrastructure, cultivating specialized talent, and ensuring high-quality data to successfully integrate AI into business operations.

3) Data Center Directions: Servers and Infrastructure for Generative AI Fuel Future Growth

The rapid adoption of generative AI is significantly impacting data center infrastructure, with global data center purchases increasing by 38% year-over-year in the first half of 2024, primarily due to AI-accelerated servers. That is according to market research from the Dell’Oro Group. The surge is expected to continue, with projections indicating a 35% rise in data center infrastructure spending, surpassing $400 billion by year-end. This growth is being driven by the necessity of investing in advanced servers and networking equipment to effectively support AI workloads.

4) Network Support for AI

AI adoption in large enterprises is placing unprecedented demands on network infrastructure, requiring IT managers to reassess their strategies for scalability, bandwidth, and latency. AI workloads, especially in model training and inferencing, generate immense data volumes that necessitate high-performance, low-latency networks capable of seamless communication between GPUs and storage systems. Technologies like Ethernet and InfiniBand are increasingly being evaluated for their ability to handle these workloads, with enhancements to Ethernet showing promise in balancing performance and cost. For IT leaders, ensuring their networks can support the demands of AI involves planning for higher-capacity hardware, advanced load balancing, and network optimization to enable efficient and cost-effective deployment of AI applications across their enterprises.

5) Dell, Deloitte, NVIDIA Roll Out New AI Factory Infrastructure

Dell Technologies, Deloitte, and NVIDIA have collaborated to introduce advanced AI Factory infrastructure solutions, aiming to streamline the deployment and management of AI workloads for large enterprises. The AI factory concept is similar to past approaches taken by the industry when trying to support HPC workloads in enterprises that normally did not need such compute capacity. Back then, vendors and solution providers offered turnkey HPC systems. Similarly, today, Dell Technologies, Deloitte, NVIDIA, and others are tightly integrating compute, storage, and networking elements in a way that optimizes an entire system for AI workloads.

A Final Word on Infrastructure for AI

Satisfying AI’s infrastructure requirements will be a constant issue in 2025 and years to come. There will be a steady stream of new solutions and innovations from leading vendors and industry groups such as the Ultra Ethernet Consortium.

Follow our coverage of infrastructure for AI to keep up to date with these developments.