Search papers, labs, and topics across Lattice.
This paper introduces an automated approach for assessing teachers' geometric reasoning skills based on the Van Hiele model, which characterizes geometric understanding through hierarchical levels. The core innovation involves integrating a structured skills dictionary, decomposing Van Hiele levels into 33 fine-grained reasoning skills, into both retrieval-augmented generation (RAG) and multi-task learning (MTL) classification models. Results demonstrate that skills-aware RAG and MTL models significantly outperform baselines in classifying teachers' Van Hiele levels from open-ended responses.
LLMs can now automatically diagnose teachers' geometric reasoning skills with higher accuracy by explicitly modeling fine-grained geometric reasoning skills, paving the way for scalable teacher training.
Assessing teachers'geometric content knowledge is essential for geometry instructional quality and student learning, but difficult to scale. The Van Hiele model characterizes geometric reasoning through five hierarchical levels. Traditional Van Hiele assessment relies on manual expert analysis of open-ended responses. This process is time-consuming, costly, and prevents large-scale evaluation. This study develops an automated approach for diagnosing teachers'Van Hiele reasoning levels using large language models grounded in educational theory. Our central hypothesis is that integrating explicit skills information significantly improves Van Hiele classification. In collaboration with mathematics education researchers, we built a structured skills dictionary decomposing the Van Hiele levels into 33 fine-grained reasoning skills. Through a custom web platform, 31 pre-service teachers solved geometry problems, yielding 226 responses. Expert researchers then annotated each response with its Van Hiele level and demonstrated skills from the dictionary. Using this annotated dataset, we implemented two classification approaches: (1) retrieval-augmented generation (RAG) and (2) multi-task learning (MTL). Each approach compared a skills-aware variant incorporating the skills dictionary against a baseline without skills information. Results showed that for both methods, skills-aware variants significantly outperformed baselines across multiple evaluation metrics. This work provides the first automated approach for Van Hiele level classification from open-ended responses. It offers a scalable, theory-grounded method for assessing teachers'geometric reasoning that can enable large-scale evaluation and support adaptive, personalized teacher learning systems.
