Search papers, labs, and topics across Lattice.
This paper investigates hallucination detection in black-box large language models (LLMs) by leveraging two complementary signals: semantic entropy and token log-probabilities. The authors develop and evaluate several methods, including the hybrid CoCoA method and two supervised approaches鈥擥ated and Stacked鈥攁cross multiple benchmarks, finding that Stacked outperforms other methods in nearly half of the cases. The results highlight the importance of combining different detection signals to improve the reliability of LLMs in high-stakes applications, while also revealing the need for careful calibration of thresholds in methods without supervised training labels.
Stacked, a novel supervised method, significantly enhances hallucination detection in LLMs, outperforming alternatives in nearly half of the tested scenarios.
When LLMs support public-facing or high-stakes workflows, missed fabrications can harm users and institutions, while false alarms consume limited human-review capacity. When no trusted context or reference document is available, we study two signals accessible through black-box model APIs: semantic entropy, which measures disagreement among sampled response meanings, and uncertainty derived from token log-probabilities. Their failure modes can be complementary: semantic entropy becomes uninformative when responses form one semantic cluster, while token uncertainty can miss consistently confident errors. We extend token-based uncertainty detection by aggregating token-level signals across sampled responses through our TopK method, evaluate the hybrid CoCoA method, which combines target-response uncertainty with semantic dissimilarity, and propose and study two supervised methods: Gated, which routes single-cluster cases to an aggregated-token-feature classifier, and Stacked, which learns jointly from semantic uncertainty and broader token features. We evaluate seven benchmarks, including five public benchmarks (four text datasets and multimodal handwritten-cheque extraction) and two constructed benchmarks (Financial Summaries and Long-Text QA), using four language models. In our evaluation across models and datasets, Stacked gave the best performance in nearly half of the cases, while TopK and CoCoA remain competitive without supervised training labels, although their thresholds require careful calibration. No method is universally strongest. We therefore evaluate performance at false-positive-rate budgets from 1% to 15%, assess their sensitivity to generation and calibration choices, and examine variation across dataset characteristics.
