Search papers, labs, and topics across Lattice.
UA-ChatDev introduces an uncertainty-aware framework for multi-agent software development that integrates uncertainty quantification into agent interactions to mitigate the risks of hallucination propagation. By employing a lightweight mechanism based on token-level log probabilities and phase-aware threshold calibration, the framework selectively triggers verification processes when agent confidence is low. Extensive experiments show that UA-ChatDev significantly improves software quality metrics, including completeness and executability, compared to existing frameworks.
Uncertainty-aware interactions in UA-ChatDev enhance code execution reliability, outperforming traditional multi-agent frameworks in software development.
Software development is a complex task that demands cooperation among agents with diverse roles. Large language models (LLMs) have enabled autonomous multi-agent software development frameworks that leverage role-based collaboration to automate requirements analysis, coding, testing, and refinement. However, existing approaches typically assume that intermediate agent outputs are equally reliable, leaving them vulnerable to hallucination propagation, where incorrect decisions generated in early development phases are transferred to downstream agents and negatively impact final software quality. To address this challenge, we propose UA-ChatDev, an uncertainty-aware multi-agent software development framework that integrates uncertainty quantification into agent interactions. It introduces a lightweight uncertainty estimation mechanism based on token-level log probabilities to assess the confidence of agent responses and employs phase-aware threshold calibration to selectively trigger retrieval-based verification when uncertainty exceeds acceptable levels. Extensive experiments on the SRDD benchmark demonstrate that UA-ChatDev consistently outperforms existing single-agent and multi-agent software development frameworks across completeness, executability, consistency, and overall quality metrics. Further ablation studies and communication analyses verify that uncertainty-aware interactions enhance code execution reliability.
