Epoch AI launched its Automation Reports on October 8, 2026, to evaluate how well frontier AI models can perform the organization's internal research tasks. The benchmark consists of 11 distinct assignments across five categories, graded by humans using internal quality rubrics.
Anthropic’s Claude Fable 5.1 and OpenAI’s GPT-6 Astra tied for the top position, each achieving an average score of approximately 65%. Other models included Grok 4.6 at 59%, Qwen 3.8 Max at 53%, Kimi K3 at 52%, and Gemini 3.8 Flash at 42%.
The evaluation indicates that while models excel at structured tasks like coding and computational analysis, they struggle with open-ended assignments. Specifically, the models failed to consistently adhere to house styles, framing, and content targeting requirements.
The Automation Reports serve as a supplement to the existing Epoch Capabilities Index. Additionally, on October 7, 2026, the organization introduced InnovationEval, a separate benchmark designed to test the ability of AI to automate the research process itself.
