Search papers, labs, and topics across Lattice.
This study introduces Multi-source Evidence Consensus Verification (MECV), a framework designed to enhance factual reliability in AI-generated news summaries by aggregating evidence from diverse sources such as source documents, Wikipedia, and open-web retrieval. The framework employs a multi-LLM jury mechanism that assesses factual reliability through contradiction-aware consensus scoring, allowing for the iterative refinement of unsupported claims. Evaluated on the SummEdits benchmark, MECV significantly improves factual consistency while maintaining the semantic integrity of the original summaries, highlighting the effectiveness of consensus across heterogeneous sources in identifying factual uncertainty.
Aggregating evidence from multiple sources can dramatically enhance the factual reliability of AI-generated news summaries, addressing a critical challenge in automated journalism.
This study proposes Multi-source Evidence Consen- sus Verification (MECV), a post-hoc hallucination cor- rection framework for AI-generated news summariza- tion. Instead of depending on a single retrieval channel, MECV aggregates evidence from multiple heterogeneous sources, including the source document, Wikipedia, and open-web retrieval. The framework further incorporates a multi-LLM jury mechanism that estimates factual reliabil- ity through contradiction-aware consensus scoring across verifier models. Claims identified as potentially unsup- ported are revised through iterative minimal-edit refine- ment. The proposed framework is evaluated on the SummEd- its benchmark using GPT-4o-mini and DeepSeek-Chat as the verifier jury, with Qwen-Plus as the orchestra- tor. Experimental results show that MECV improves fac- tual consistency while preserving the semantic structure of the original summaries. The findings further suggest that agreement across heterogeneous evidence sources can serve as a useful signal for identifying factual uncertainty in AI-generated summaries, including in information- sensitive domains such as financial news aggregation. This study contributes to research on trustworthy AI and automated journalism by introducing a multi-source verification framework for hallucination correction and demonstrating the value of consensus-based verification for improving factual reliability in AI-generated news summarization.
