Search papers, labs, and topics across Lattice.
This paper introduces a method for training open-weight language models to simulate student programming behavior by serializing student interaction logs into conversational dialogues between the student and the automated assessment system. Supervised fine-tuning and preference optimization are used to align the models with authentic debugging behavior. Experiments training Qwen models (4B and 8B) on a large Python dataset demonstrate that incorporating environment feedback significantly improves the models' ability to replicate student debugging, outperforming code-only approaches and prompted LLMs.
Open-weight language models can now mimic student debugging processes with surprising fidelity by learning from conversational logs of student-environment interactions, offering a privacy-respecting alternative to prompting proprietary LLMs.
Artificial models that simulate how learners act and respond within educational systems are a promising tool for evaluating tutoring strategies and feedback mechanisms at scale. However, many existing approaches in programming education rely on prompting large, proprietary language models, raising concerns around privacy, cost, and dependence. In this work, we propose a method for training open-weight artificial programming learners using authentic student process data. Our approach serializes temporal log traces into a conversational format, representing each student's problem-solving process as a dialogue between the learner and their automated assessment system. Student code submissions and environment feedback, such as test outcomes, grades, and error traces, form alternating conversational turns, enabling models to learn from the iterative debugging process. We additionally introduce a training pipeline combining supervised fine-tuning with preference optimization to align models with authentic student debugging behavior. We evaluate our framework by training Qwen models at 4B and 8B scales on a large-scale dataset of real student submissions to Python programming assignments. Our results show that incorporating environment feedback strengthens the models' ability to replicate student debugging behavior, improving over both prior code-only approaches and prompted large language models baselines in functional alignment and code similarity. We release our code to support reproducibility.
