Search papers, labs, and topics across Lattice.
This paper introduces CAPRI, a contract-aware proof repair workflow designed to enhance the reliability of large language models (LLMs) in generating Isabelle proofs. By implementing a machine-readable edit contract alongside Isabelle's proof-checking capabilities, CAPRI ensures that only authorized changes are made, significantly reducing the risk of unintended modifications. The evaluation of five workflows on twelve failed proofs demonstrated that CAPRI achieved a high rate of valid repairs while maintaining contract integrity, with a notable performance difference across various iterative and one-shot repair approaches.
CAPRI reveals that a contract-aware approach can drastically improve the integrity of LLM-generated proofs, achieving up to 81% valid repairs without violating edit contracts.
We address the use of large language models (LLMs) to help discover Isabelle proofs. An Isabelle build establishes that the submitted theory is accepted, but not that an LLM changed only what the developer authorised. We present CAPRI, a contract-aware repair workflow in which Isabelle checks the proof and an independent checker enforces a machine-readable edit contract. Prompts, proposals, candidate repositories, diagnostics, verdicts, and hashes are retained for audit. We evaluate five workflows on twelve failed proofs from four developments, with three replicates per task and condition, giving 180 runs and 138 valid repairs. Of 144 terminal candidates accepted by Isabelle, six had modified protected text; all arose in iterative workflows that could edit a complete theory. A proof-body-only interface produced 29/36 valid repairs and no contract violations, compared with 31/36 for the corresponding full-theory workflow. One-shot repair produced 22/36, while a later prospectively frozen iterative workflow produced 32/36; these figures compare complete workflows rather than individual mechanisms. A separate post hoc OpenRouter campaign found no improvement in the designated Luna comparisons. A Sol configuration with matched demonstrations produced 33/36 repairs, compared with 29/36 in the frozen OpenAI Responses condition, but the difference was not statistically significant in a one-sided exact McNemar test ($p=0.0625$).
