Search papers, labs, and topics across Lattice.
This study evaluates the performance of various AI systems on ten research-level mathematics problems sourced from active mathematicians. By analyzing the AI-generated solutions against human solutions and referee reports, the authors assess the capabilities and limitations of current AI in tackling complex mathematical challenges. The findings reveal significant gaps in AI performance, highlighting areas for improvement in mathematical reasoning and problem-solving skills.
Current AI systems struggle with research-level mathematics, often failing to match human problem-solving capabilities on complex tasks.
To assess the ability of current AI systems to correctly solve research-level mathematics problems, we tested several AI systems on a set of ten problems in a broad range of mathematical fields; these problems arose naturally in the research process of the contributors. This document includes the problems, our methodology, and the results of our testing. We provide links to supplementary documents including the human solutions, the AI-generated solutions, and the referee reports and logs for the AI-generated solutions. The ten problems were contributed by the following mathematicians: (1) Dariusz Kaloci艅ski and Theodore A. Slaman, (2) Richard Schwartz, (3) Aleksa Milojevic and Benny Sudakov, (4) Larry Guth, (5) Oleg Butkovsky, Jonathan Mattingly, and Lorenzo Zambotti, (6) Joshua Evan Greene and Duncan McCoy, (7) Sucharit Sarkar, (8) Sam Payne and Jidong (Jayden) Wang, (9) Sylvie Corteel and John Lentfer, (10) Srivatsav Kunnawalkam Elayavalli.
ETH