According to Epoch AI, the total capacity of hardware for agentic inference increased by 4.6 times between the second quarter of 2025 and the second quarter of 2026. The peak performance in 8-bit operations grew by 2.6 times during the same period. Researchers estimate a 90% probability that the growth in inference capacity falls within the range of 4.2 to 5.1 times.
Epoch AI utilizes the Inference Performance Index (IPI) to evaluate hardware. This metric considers memory capacity, bandwidth, and data transfer speeds alongside raw computing power. Under this model, a GB300 chip provides 8 times more inference capacity than an H200. However, in real-world tests with the Kimi K3 model, the actual throughput of the GB300 reached only about half of its calculated potential.
Nvidia dominates the market, with Blackwell and Blackwell Ultra families accounting for 73% of total agentic inference capacity by mid-2026. Specifically, B300 and GB300 systems provide 47% of the total, while B200 and GB200 contribute 26%. Google TPU v7 accelerators hold a 14% share. The disparity between inference capacity and peak computing power highlights the critical role of architecture in system efficiency.
The report suggests a future slowdown in growth. New chips sold in the second quarter of 2026 added 3.3 times more agentic capacity compared to those delivered a year earlier, which is a lower rate than the growth of the total installed base. Looking ahead, the transition from GB300 to the Vera Rubin VR200 platform is expected to yield a performance increase of approximately 2.2 times.



