Search papers, labs, and topics across Lattice.
The authors introduce LiveK12Bench, a dynamic benchmark of 2K+ questions across STEM disciplines sourced from real-world high school exams, designed to evaluate LMMs in realistic testing environments. The benchmark features an automated pipeline to prevent data leakage and a "Mock Exam" evaluation scheme to assess end-to-end exam-taking ability. Experiments on 12 LMMs show significant performance drops under realistic constraints, with GPT-4's score decreasing from 79 to 53 when process rigor and efficiency are jointly evaluated, revealing vulnerabilities in handling complex visual layouts.
Even GPT-4 struggles to maintain its reasoning prowess when faced with the rigor and efficiency demands of a realistic high school exam, suggesting current LMMs are far from being ready for prime time as intelligent tutors.
Advanced Large Multimodal Models (LMMs) have demonstrated impressive performance in K-12 reasoning tasks, exhibiting great promise as intelligent tutors. Realizing this potential requires models to navigate real-world examinations effectively, yet most existing benchmarks fail to capture the complexity of authentic testing environments. Specifically, most datasets are static, prone to data contamination, and are often confined to restricted modalities, disciplines, and evaluation criteria. To address these issues, we introduce LiveK12Bench, a dynamic, holistic, multi-disciplinary benchmark designed to evaluate the reasoning abilities of LMMs in realistic examination scenarios. LiveK12Bench comprises 2K+ verified questions spanning Mathematics, Physics, Chemistry, and Biology, sourced from the latest real-world exam papers and designed to grow over time. Our framework features several core innovations: 1) featuring an automated pipeline that continuously ingests and parses the latest examination papers to mitigate data leakage; and 2) proposing a novel `Mock Exam'evaluation scheme, which assesses the ability to complete end-to-end exams autonomously with accurate and efficient reasoning paths. Extensive experiments on 12 LMMs reveal that advanced models suffer substantial performance degradation under exam-realistic constraints: GPT-5's score drops from 79 to 53 (out of 100) when process rigor and efficiency are jointly evaluated. Our findings expose critical vulnerabilities, such as sensitivity to complex visual layouts, highlighting the gap between idealized reasoning capabilities and true educational readiness. Both code and dataset are publicly available.
