Search papers, labs, and topics across Lattice.
This paper addresses the challenges of generating Verilog code using Large Language Models (LLMs) by introducing the Execution-Anchored Hallucination Calibration (EAHC) reranking framework. The authors identify critical limitations in existing methods, including poor domain transferability of execution-based reranking and the inconsistency of LLM-as-a-Judge due to reasoning hallucinations. By independently acquiring execution and reasoning signals and combining them at the decision stage, EAHC improves the consistency of scores for execution-equivalent code candidates, demonstrating enhanced performance across multiple models and benchmarks.
Execution signals and reasoning signals have orthogonal errors, and their independent calibration can significantly enhance Verilog code generation accuracy.
Large Language Models (LLMs) have demonstrated remarkable capabilities in code generation, yet their performance degrades significantly on low-resource Hardware Description Languages such as Verilog. While multi-candidate sampling improves the likelihood of generating correct solutions, au-tomatically selecting the optimal candidate remains an open challenge. Through a systematic empirical study across nine models and two benchmarks, we identify two critical limitations:(1) existing execution-based reranking methods, which rely on testbench pass/fail outcomes, exhibit poor domain transferability due to low-quality generated testbenches; and (2) LLM-as-a-Judge suffers from reasoning hallucination, producing incon-sistent judgments for execution-equivalent code. These findings reveal two signal types with orthogonal errors: execution signals(deterministic but testbench coverage limited)and reasoning signals (semantically rich but hallucination-prone). Their orthog-onality suggests combining the two signals, yet in our experiments letting the reasoner directly observe execution results merely anchors its judgments on test outcomes; we therefore acquire the two signals independently and fuse them only at the decision stage. Based on these insights, we propose EAHC, an Execution-Anchored Hallucination Calibration reranking framework that anchors reasoning judgments to execution behavior so that execution-equivalent candidates receive consistent scores, which implements a dual-channel architecture: EAHC-R, a 4B reasoning discriminator; and EAHC-T, a testbench generator leveraging RAG for execution verification.
