The TheoPhysEval benchmark, published on arXiv on October 9, 2026, evaluates AI agents on 50 unpublished theoretical physics problems. The study, involving 35 researchers from institutions including ICTS and Google DeepMind, tested off-the-shelf models across 11 subfields, such as quantum field theory and fluid dynamics.
Testing was conducted in two modes: realistic, allowing web access, and isolated, without internet connectivity. The GPT-6-Astra model solved 49 out of 50 problems in the realistic track at a cost of $116.28. In the isolated track, its performance decreased to 46 solved problems.
The Opus 5.5 model achieved a score of 46 out of 50 in both testing modes. Across all models tested, one specific problem remained unsolved. The study noted that AI outputs led to corrections in some human-written reference solutions.
The researchers emphasized that using unpublished problems reduces the likelihood of models relying on training data recall. While the study highlights autonomous problem-solving capabilities, it notes that the benchmark does not assess the ability to frame new research questions.
