Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The authors evaluate the ability of six LLMs to simulate subjective student evaluations of educational feedback on high-school biology questions across both individual and aggregate levels. Understanding this capability is critical as LLM-as-a-judge pipelines and synthetic persona-based testing increasingly substitute for real human subjects in personalized education and alignment workflows. The results show that while conditioning models on student personality traits and few-shot evaluation examples improves individual score calibration, it consistently fails to recover authentic group-level preference distributions.
Personalizing LLMs with student profiles improves individual score calibration, yet counterintuitively fails to capture aggregate group-level preferences in educational evaluation.
While recent studies have explored human behavior and preference simulation using large language models (LLMs), it remains unclear how well LLMs can simulate subjective evaluations from real learners in educational settings. We investigate this question using real learner evaluation data on feedback for high-school biology questions at both the group and individual levels. We compare performance with and without learner-specific information, such as personality traits and evaluation examples, across six models. Our results show that LLMs still have a limited ability to simulate learner evaluations. Providing learner profiles and examples improves score calibration and individual-level simulation, but more often fails to improve group-level consistency. These findings highlight the need to investigate which learner information and adaptation strategies are effective for learner preference simulation.