Search papers, labs, and topics across Lattice.
This study addresses the issue of automated feedback systems reinforcing misconceptions when students arrive at correct answers through flawed reasoning. By analyzing 20,964 student responses from the Eedi mathematics platform, the researchers found that while fine-tuned classifiers detected only 57% of hidden misconceptions, an open-weight reasoning model improved detection to 84%, albeit with a high false alarm rate. The authors propose a graduated assessment rubric and a detect-verify-escalate pipeline to improve feedback accuracy and provide targeted follow-up for uncertain cases, enhancing the learning process.
Hidden misconceptions in student responses can be detected with 84% accuracy, but the cost of false alarms is alarmingly high—8 to 1 against genuine detections.
Automated feedback systems that rely on answer correctness will reinforce, rather than address, misconceptions when students reach the correct answer through flawed reasoning. We investigate automatic detection of these hidden misconceptions using 20,964 real student responses from the Eedi mathematics platform. Fine-tuned classifiers detect only 57% of these hidden misconceptions, and standard ML interventions do not improve on this. An open-weight reasoning model detects 84%, but at realistic prevalence, false alarms outnumber genuine detections roughly 8 to 1. We present a graduated assessment rubric that separates answer correctness from method validity, and propose a detect-verify-escalate pipeline that routes uncertain cases to diagnostic follow-up questions rather than directly to teachers. Two deployment modes adapt the pipeline: a teacher dashboard where the system filters a review queue, and an autonomous tutor where flags trigger low-cost formative follow-up.
