Search papers, labs, and topics across Lattice.
This paper addresses the critical issue of hallucinations in LLM-based timeline summarization (TLS) by proposing NTS-CoT, a framework that employs Chain-of-Thought (CoT) reasoning to enhance the fidelity of generated summaries. The authors identify two main types of hallucinations—unfaithful content and information omission—and introduce three modules within NTS-CoT: Element-CoT for capturing essential news elements, Date Selection for optimal timestamping, and Causal-CoT for inferring causal relationships. Experimental results show that NTS-CoT significantly outperforms existing baselines, effectively reducing hallucinations and improving the overall performance of LLM-based TLS.
Hallucinations in LLM-generated news summaries can be effectively mitigated using a novel Chain-of-Thought reasoning framework, leading to more accurate and reliable timeline summarization.
The rapid updates of online news make tracking event developments challenging, highlighting the need for timeline summarization (TLS). Hallucinations, where LLM-generated content deviates from source news, still remain a critical issue in LLM-based TLS and are not well studied in existing works. To bridge this gap, we identify two primary types of hallucinations: unfaithful content during news summarization and information omission in date-event summarization. Then, we propose NTS-CoT, a novel framework that leverages Chain-of-Thought (CoT) reasoning to mitigate hallucinations in TLS. The framework consists of three key modules: i) Element-CoT to capture essential news elements for faithful summarization, ii) Date Selection to combine temporal saliency and event prominence for timestamp selection, and iii) Causal-CoT to infer causal relationships and reduce omissions in date-event summarization. Extensive experiments, including quantitative analysis on three TLS benchmarks and human evaluation, demonstrate that NTS-CoT outperforms state-of-the-art baselines, effectively mitigating hallucinations and improving LLM-based TLS performance. Our source code is available at https://anonymous.4open.science/r/NTS-CoT .