Sakana AI has developed Multi-Layered Review (MLR), an agentic system designed to critique research papers. The process utilizes three agents powered by Claude models to generate outlines, identify weaknesses, and synthesize final feedback including scores and recommendations. The system processes PDF files directly, preserving figures and equations.
To evaluate the system, researchers created a benchmark using 257 papers from major conferences like NeurIPS and ICML. By injecting contradictions into knowledge graphs, the team generated 1,164 data points. MLR outperformed existing baselines, detecting 73.43% of core-claim errors with four reviews, compared to 14.81% by the AgentReview baseline.
The system demonstrated high accuracy on clean papers and showed a 0.586 Pearson correlation with human scores on ICLR 2025 submissions. While MLR focuses heavily on experimental validity, it remains a complementary tool rather than a human replacement. The authors note that human reviewers prioritize novelty and clarity.
Operationally, MLR costs approximately $0.47 per review and consumes 189,062 input tokens. This efficiency is significantly higher than the AI Reviewer baseline, which requires 403,654 tokens. A simplified single-prompt version of the system can further reduce costs by two-thirds, though it results in a slight decrease in detection performance.
