At the AI Infra Summit in Santa Clara, industry leaders including Amazon Web Services, Oracle, Broadcom, Qualcomm, and d-Matrix discussed strategies to optimize infrastructure for AI workloads. Experts highlighted that the shift toward inference-heavy tasks is driving the need for heterogeneous compute architectures that combine GPUs with CPUs.
AWS continues to expand its Graviton CPU line, launching the Graviton5 in June to improve energy efficiency for real-time AI reasoning. Meanwhile, memory remains a critical bottleneck. Analysts project that spending on AI server memory will grow from $35 billion in 2025 to between $175 billion and $190 billion by 2027.
Qualcomm is promoting its proprietary high-bandwidth compute architecture, claiming it provides six times the bandwidth per watt compared to HBM. Startup d-Matrix is pursuing a different approach by integrating 3D DRAM into its Raptor chips, a technology recently incorporated into Nvidia’s NVLink Fusion reference architecture for high-performance deployments.
Networking technology is also evolving to support large-scale AI clusters. Oracle has introduced its Acceleron architecture to optimize performance, while Broadcom is leveraging Ethernet fabric and its Tomahawk 6 chip with Cognitive Routing 2.0 to mitigate data congestion. These developments aim to address the rising costs and performance demands of enterprise AI.
