Epoch AI has introduced the Inference Performance Index (IPI) Version 1, designed to measure agentic inference capacity rather than relying solely on traditional FLOP/s metrics. The index evaluates systems based on their ability to handle concurrent requests at speeds ranging from 50 to 400 tokens per second.

The methodology accounts for memory bandwidth, inter-chip communication, and compute capability. Results are normalized to the Nvidia GB300 NVL72, which serves as the baseline with a score of 1.0 GB300e. The Nvidia GB300 demonstrates 25 times the inference capacity of the H100, despite having only 2.5 times the eight-bit FLOP/s.

Current rankings show the MI455X leading with a score of 2.89, followed by the VR200 at 2.18. The H200 HGX achieves approximately one-eighth of the GB300 NVL72 capacity, while the MI355X and B300 HGX show nearly identical performance metrics.

Data from November 18 indicates that total agentic inference capacity increased 4.6 times between Q2 2025 and Q2 2026, significantly outpacing the 2.6-fold growth in raw compute power. Nvidia’s Blackwell GPU series accounted for 74% of this capacity expansion.

The index utilizes Nvidia’s AISimulate projections and is available as open-source code on GitHub. This allows independent researchers and competitors to verify the performance curves and evaluate how memory and interconnect factors influence real-world AI workloads.