Search papers, labs, and topics across Lattice.
This study analyzes 7,201 Prolog submissions from undergraduate students to create a detailed taxonomy of common student bugs, derived from a manual classification of 200 bug-fixing submissions. Using this taxonomy, the authors developed LogMorph, a data-driven mutation tool that generates realistic faults by weighting mutation operators according to the observed error distribution in student submissions. Evaluation of 16,000 mutants demonstrates that LogMorph's synthetic error distribution closely mirrors actual student errors, with most categories aligning within two percentage points, highlighting the tool's effectiveness in enhancing automated feedback mechanisms in logic programming education.
LogMorph generates realistic Prolog bugs that reflect actual student mistakes, improving automated feedback tools in logic programming education.
Automated feedback tools for logic programming education depend on realistic bug datasets that reflect the mistakes students actually make. However, existing mutation testing frameworks for Prolog treat all mutations as equally likely, producing synthetic faults that diverge from classroom reality. We present an empirical study of 7,201 Prolog submissions from 265 undergraduate students, from which we derive a fine-grained taxonomy of student bugs through manual classification of 200 bug-fixing submissions. Guided by this taxonomy, we develop LogMorph, a data-driven mutation tool whose 17 operators are weighted according to the observed error distribution. LogMorph enumerates valid mutation sites on the abstract syntax tree, samples operators proportionally, injects faults, delegating to an SMT-based synthesizer when new code fragments are needed, and validates each mutant against a reference test suite. An evaluation of 16,000 generated mutants shows that the synthetic error distribution closely matches the student distribution, with most bug categories agreeing to within two percentage points. We identify cut-related mutations and synthesizer-generated code as the main sources of residual divergence, and outline how combining the SMT back-end with a language model fine-tuned on student code can further improve realism.
