Microsoft has unveiled the Maia 300, a new AI inference accelerator designed for internal use. The chip utilizes a multichiplet architecture, featuring four computing chiplets manufactured on a two-nanometer process by TSMC. It is equipped with eight HBM4 modules providing 288 gigabytes of memory capacity.
The architecture focuses on minimizing data movement to improve efficiency. Computing chiplets prioritize the use of internal SRAM cache, while data sent to HBM stacks is routed to physically adjacent modules. The chip also supports data compression formats like MXFP4 to accelerate internal processing.
Microsoft plans to deploy the accelerators in liquid-cooled racks, each containing 96 units. These racks are expected to deliver seven times the token throughput compared to the previous Maia 200 generation. The company intends to organize these racks into superpods containing 12,288 accelerators.
The system uses a custom Ethernet-based network stack that allows for flexible bandwidth allocation between in-rack and inter-rack traffic. By using consistent communication semantics, the design eliminates the need for additional host server hardware to manage data exchange between different network modes.
Production deployment of the Maia 300 is scheduled to begin next year. Microsoft intends for this custom silicon to serve as its primary source of inference capacity, supporting larger models and more concurrent sessions within its data centers.
