Search papers, labs, and topics across Lattice.
This paper introduces Fast Adaptive Semantic Entropy (FASE), a new metric designed to enhance the reliability of multi-agent code generation by approximating functional correctness through structural and semantic dissimilarity graphs. FASE significantly improves upon existing semantic entropy methods by achieving a 25% increase in Spearman correlation and a 19% boost in ROCAUC score compared to ground-truth test cases, all while dramatically reducing computational costs to just 0.3% of traditional methods. These advancements address the challenges of LLM hallucinations and error propagation, making FASE a viable solution for autonomous software development.
FASE achieves a 25% improvement in correlation with ground-truth test cases while slashing computational costs to a fraction of traditional methods.
Multi-agent code generation offers a promising paradigm for autonomous software development by simulating the human software engineering lifecycle. However, system reliability remains hindered by LLM hallucinations and error propagation across interacting agents. While semantic entropy provides a principled way to quantify uncertainty without ground-truth answers, current methods often rely on costly LLM-driven equivalence checks. In this work, we introduce Fast Adaptive Semantic Entropy (FASE), a novel metric that approximates functional correctness based on the minimum spanning tree of structural and semantic dissimilarity graphs. Evaluations on HumanEval and BigCodeBench demonstrate that FASE outperforms state-of-the-art semantic entropy by LLM entailment, achieving a 25% average improvement in Spearman correlation and a 19% increase in ROCAUC score against Pass@1 from ground-truth test cases when using the Qwen3-Embedding-8B model. Furthermore, by eliminating costly LLM-driven equivalence evaluation, FASE incurs negligible computational overhead, requiring only approximately 0.3% of the runtime cost of traditional semantic entropy approaches. These results position FASE as a practical, cost-effective solution for optimizing uncertainty quantification in real-world multi-agent workflows.
