The new benchmark evaluates CPU performance in agentic AI workloads by measuring throughput under concurrent conditions. The methodology involves running multiple agent sessions simultaneously to identify the point where resource contention impacts processing efficiency.
The test uses custom agents built with LangGraph, which execute a sequence of 30 tasks. To ensure consistency, the system utilizes a proxy server that records traces of LLM interactions and tool calls. During testing, these recorded traces are replayed, allowing the benchmark to bypass live model inference and focus on CPU-bound operations.
The benchmark scales from a single worker process up to the maximum thread count of the CPU. The primary metric is sessions per second, which tracks how many tasks are completed as concurrency increases. The test produces two key scores: peak throughput and ideal throughput.
Ideal throughput is defined as the point where the slowest 5% of sessions do not exceed 1.5 times the duration of a single session at base concurrency. This approach helps determine the optimal number of agent sessions a processor can handle while maintaining interactive performance.
