Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces Classroom Final Exam (CFE), a new multimodal benchmark comprising real-world university STEM exam questions with instructor-provided solutions, designed to evaluate the reasoning capabilities of large language models. CFE poses a significant challenge to current state-of-the-art models, with Gemini-3.1-pro-preview achieving only 59.69% accuracy. Diagnostic analysis reveals that models struggle with maintaining correct intermediate states in multi-step reasoning and exhibit suboptimal solution efficiency compared to instructor solutions.
Even the best LLMs struggle to maintain correct intermediate states when solving university-level STEM problems, often taking more steps than necessary and accumulating errors along the way.
We introduce \CFE{} (\textbf{C}lassroom \textbf{F}inal \textbf{E}xam), a multimodal benchmark for evaluating the reasoning capabilities of large language models across more than 20 STEM domains. \CFE{} is curated from repeatedly used, authentic university homework and exam problems, together with reference solutions provided by course instructors. \CFE{} presents a significant challenge even for frontier models: the newly released Gemini-3.1-pro-preview achieves an overall accuracy of 59.69\%, while the second-best model, Gemini-3-flash-preview, reaches 55.46\%, leaving considerable room for improvement. Beyond leaderboard results, we perform a diagnostic analysis by decomposing reference solutions into reasoning flows. We find that although frontier models can often answer intermediate sub-questions correctly, they struggle to reliably derive and maintain correct intermediate states throughout multi-step solutions. We further observe that model-generated solutions typically have more reasoning steps than those provided by the instructor, indicating suboptimal step efficiency and a higher risk of error accumulation. The data and code are available at https://github.com/Analogy-AI/CFE_Bench.
CMU ML