Search papers, labs, and topics across Lattice.
PennySynth, a retrieval-augmented generation (RAG) framework, was developed to address the limitations of general-purpose LLMs in generating correct PennyLane quantum code. It conditions LLM inference on a curated knowledge base of 13,389 PennyLane instruction-code pairs, constructed through a three-stage pipeline. PennySynth significantly outperforms Claude Sonnet 4.6 on QHack challenges, achieving up to a 28 percentage point improvement in pass@5, demonstrating the effectiveness of code-aware embeddings and RAG for specialized code generation.
General-purpose LLMs can't hack it in quantum coding, but a RAG system armed with a code-aware embedding strategy can boost performance by 28% on real-world quantum programming challenges.
The growing complexity of quantum programming frameworks has exposed a critical limitation in existing large language model (LLM)-based code assistants: general-purpose models hallucinate PennyLane-specific gate names, misplace device configurations, and produce structurally invalid circuits when faced with specialized quantum coding challenges. We present PennySynth, a retrieval-augmented generation framework that addresses this gap by conditioning LLM inference on a curated knowledge base of 13,389 PennyLane instruction-code pairs, built via a three-stage extraction, verification, and deduplication pipeline over official PennyLane repositories, community GitHub sources, and QHack competition archives. PennySynth introduces a code-aware embedding strategy using st-codesearch-distilroberta-base, trained for natural-language-to-code retrieval, increasing average retrieval cosine similarity from 0.45 to 0.726 compared to a general-purpose baseline. Evaluated across 74 challenges spanning three years of the QHack competition (2022, 2023, 2024), PennySynth achieves 64%, 68%, and 52% pass@5 on QHack 2022, 2023, and 2024, respectively, improving over Claude Sonnet 4.6 without retrieval by +28, +25, and +28 percentage points. We further introduce a quantum-adapted CodeBLEU metric that upweights qml.* token patterns and show that structural code similarity and functional correctness capture distinct aspects of quantum code quality. Controlled ablations reveal that code-aware embeddings are the primary driver of retrieval performance, while dataset expansion and source composition provide additional gains when retrieval quality is sufficiently precise.
