OpenAI recently published hundreds of solutions to complex mathematical problems, claiming to have consulted an elite advisory group to ensure quality. However, the Advisory Group on Mathematics and Artificial Intelligence (AGMAI) noted that OpenAI ignored key recommendations, including the request to stop testing advanced problems on proprietary models.
The AGMAI, which includes nine global researchers, established guidelines for frontier labs in late September. OpenAI failed to follow several principles, such as providing chain-of-thought data for all results; only 10 out of 719 manuscripts included this information. Additionally, 42% of the released proofs were not formalized.
A new paper from researchers at the University of Cambridge and King’s College London identified discrepancies between natural language proofs and Lean code in OpenAI’s solutions for Navier-Stokes equations. The authors argue that these auto-formalized proofs require the same level of peer review as human-generated work.
Mathematicians, including Terence Tao and Melanie Wood, emphasize that AI-generated solutions lack human understanding. They suggest that OpenAI should fund human mathematicians to verify results and engage with the scientific community, rather than treating the output as a final product.
