Search papers, labs, and topics across Lattice.
ProDebug combines LLMs with spectrum-based and mutation-based fault localization to automatically debug Prolog assignments. It detects faults using spectrum-based, mutation-based, and LLM reasoning approaches, and generates repairs using mutation-based techniques and LLMs. Evaluated on 1499 student submissions, ProDebug shows the potential of LLM-augmented feedback systems to scale support for declarative programming education.
LLMs can now automatically debug Prolog code, offering personalized feedback at scale for logic programming education.
Prolog is a well-known declarative programming language commonly used in introductory courses on logic and reasoning. However, many students find Prolog challenging because it lacks the familiar debugging mechanisms found in imperative languages. In large classes, this difficulty is exacerbated by the challenge of providing timely and personalized feedback to students. In this work, we introduce ProDebug, the first tool to combine Large Language Models (LLMs) with spectrum-based and mutation-based techniques for automated debugging of Prolog assignments. ProDebug automatically identifies faults and proposes bug repairs for student Git submissions. Faults are detected using three approaches--spectrum-based, mutation-based, and LLM reasoning--while repairs are generated using mutation-based techniques and LLMs. Our evaluation on 1499 buggy student submissions from a bachelor's level programming class demonstrates the potential of automated, LLM-augmented feedback systems to scale support for declarative programming education.
CMU ML