Search papers, labs, and topics across Lattice.
This paper addresses the challenge of generating high-quality training data for synthesizing inductive loop invariants, a bottleneck in automated program verification. They introduce Wonda, a data curation pipeline that refines noisy verifier-generated invariants using AST-based normalization, LLM-driven semantic rewriting, and augmentation with provable quality guarantees. Fine-tuning small language models (SLMs) on the curated dataset significantly improves performance, with a 4B parameter model matching the utility of a GPT-OSS-120B baseline and approaching GPT-5.2 performance on challenging instances from InvBench.
Forget massive models: fine-tuning a small 4B parameter model on carefully curated data can match or even approach the performance of 100B+ parameter LLMs in program verification tasks.
The synthesis of inductive loop invariants is a critical bottleneck in automated program verification. While Large Language Models (LLMs) show promise in mitigating this issue, they often fail on hard instances, generating invariants that are invalid or computationally ineffective. While fine-tuning is a natural route to mitigate this limitation, obtaining high-quality training data for invariant generation remains an open challenge. We present a rigorous data curation pipeline designed to extract high-quality training signals from raw verifier-generated invariants. First, we formalize the properties required for a high-quality training invariant. Second, we propose Wonda, a pipeline that refines noisy data via AST-based normalization, followed by LLM-driven semantic rewriting and augmentation with provable quality guarantees. We demonstrate that fine-tuning Small Language Models (SLMs) on this curated dataset result in consistent and significant performance gain. In particular, a fine-tuned 4B parameter model matches the utility of a GPT-OSS-120B baseline and approaches the state-of-the-art GPT-5.2, without incurring reasoning-time overhead. On challenging instances from the recent InvBench evaluation suite, our approach doubles the invariant correctness and speedup rates of base models; and improves their Virtual Best Performance (VBP) rates on the verification task by up to 14.2%.
