Search papers, labs, and topics across Lattice.
This study introduces an uncertainty-aware framework for Agentic Retrieval-Augmented Generation (RAG) systems that leverages Bayesian Networks to propagate uncertainty signals through multi-stage reasoning pipelines. By analyzing the performance of this framework on StrategyQA and HotpotQA with models like GPT-3.5-Turbo and GPT-4.1-Nano, the authors demonstrate that Bayesian propagation effectively identifies potential failure points, particularly in scenarios with accumulating uncertainty. The findings highlight the framework's strengths in HotpotQA while revealing challenges in StrategyQA due to miscalibration issues, suggesting a need for further refinement in real-world applications.
Bayesian uncertainty propagation reveals critical failure points in multi-hop reasoning, outperforming traditional methods in complex scenarios.
Trustworthy deployment of Agentic Retrieval-Augmented Generation (RAG) systems requires mechanisms for estimating when multi-stage reasoning pipelines may fail. This paper presents an uncertainty-aware Agentic Retrieval-Augmented Generation (RAG) framework in which planner, evaluator and generator stages produce uncertainty signals derived from semantic divergence and generator self-evaluation. These signals are propagated through a Bayesian Network (BN) to estimate system-level uncertainty and provide node-level indicators of potential failure points across the workflow. The approach is evaluated on StrategyQA and HotpotQA using GPT-3.5-Turbo and GPT-4.1-Nano, with Area Under the Receiver Operating Characteristic Curve (AUROC), Area Under the Accuracy-Rejection Curve (AUARC), Expected Calibration Error (ECE), and Brier Score used to assess discrimination, selective prediction and calibration. Results show that Bayesian propagation is more effective on HotpotQA, where uncertainty accumulates across multi-hop reasoning stages, while StrategyQA exposes limitations caused by miscalibration and unreliable upstream signals. The study positions Bayesian uncertainty propagation as a promising but preliminary mechanism for monitoring Agentic RAG systems, with future validation required in industrial domains such as Offshore Wind (OSW) maintenance decision support.
