Search papers, labs, and topics across Lattice.
This paper introduces HallDetect, a novel framework for detecting hallucinations in Large Language Models (LLMs) by employing a decomposition-based approach to factuality evaluation. HallDetect verifies generated content by breaking it down into atomic claims, which are assessed using a compact encoder-based entailment model and a contrastive formulation against a multi-scale library of source chunks. The framework not only outperforms existing generative and embedding-based baselines on multiple benchmarks but also provides a detailed audit trail for error localization, enhancing the reliability of LLM outputs.
HallDetect flags hallucinations by identifying just one confidently contradicted claim, revolutionizing how we ensure factual accuracy in LLM outputs.
The reliability of Large Language Models (LLMs) is often compromised by factual inconsistencies, including hallucinations---cases where generated content is not supported by the underlying source. We present HallDetect, a lightweight, reference-free, and black-box framework for hallucination detection that we evaluate not only on summarization but across a broader range of source-grounded generation settings. HallDetect builds on decomposition-based factuality evaluation: generated content is decomposed into atomic claims, each verified by a compact encoder-based entailment model through a contrastive formulation over a multi-scale library of source chunks, and aggregated with an asymmetric score in which a single confidently contradicted claim flags the response. Under a controlled protocol in which all methods share the same 4-bit quantized backbones and consumer-grade hardware budget, HallDetect outperforms comparably resourced generative and embedding-based baselines on three of four benchmarks while remaining stable across backbone families, and yields a claim-to-span audit trail that localizes each error.
