Microsoft is exploring composable memory architectures to address the growing scarcity of memory resources in AI data centers. By pooling memory capacity across servers rather than locking it within individual units, the company aims to assign resources dynamically to workloads as needed.
Principal architect Samir Rajadnya noted that memory is currently a primary bottleneck for AI inference. Microsoft is investigating system-level technologies to improve efficiency and reduce waste, ensuring that memory is available exactly where and when the workload demands it.
The strategy involves vertical integration of workloads, software, and silicon. To achieve this at scale, Microsoft is focusing on advanced interconnects, including optical solutions, to facilitate data transfer between processors and memory outside the server chassis.
The company continues to contribute infrastructure designs to the Open Compute Project. Rajadnya emphasized that progress in memory and processor components must be matched by developments in connectivity to build a truly composable infrastructure.
