Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a novel approach to hallucination detection in large language and vision-language models by leveraging density ridge selective prediction, which utilizes a kernel density estimate of hidden state generation trajectories. The method addresses the limitations of existing unsupervised and supervised detectors, particularly in scenarios with scarce calibration labels, by scoring test generations based on their proximity to the density ridge. The results show a significant improvement in AUROC scores, outperforming traditional methods by 5-20 points while maintaining robustness under label scarcity conditions.
Density ridge selective prediction achieves a 5-20 point boost in hallucination detection accuracy, even when calibration labels are scarce.
Hallucination detection in large language and vision-language models is increasingly framed as selective prediction, where a detector assigns a confidence score and abstains when confidence is low. Unsupervised sampling detectors (Semantic Entropy, EigenScore) avoid labels but plateau in quality, while supervised probes (SAPLMA) attain stronger in-distribution scores yet degrade sharply when calibration labels are scarce. We recover the response manifold of an LLM as the density ridge of a kernel density estimate built on a six-dimensional kinematic feature map of hidden state generation trajectories. A test generation is scored by the negated Euclidean distance from its projected feature point to the nearest ridge vertex, yielding a low-dimensional geometric skeleton of the stochastic output distribution. We evaluate against Semantic Entropy, SAR, EigenScore, SAPLMA, and log-probability on seven QA benchmarks (HaluEval-QA, TriviaQA, GSM8K, POPE, ScienceQA, A-OKVQA) using nine text and vision LLMs in a deliberately label-scarce protocol ($n_{\text{cal}}{=}200$ queries, $N{=}5$ generations). Our ridge-based score beats on AUROC with 5-20 points gain, while demonstrating tempered degradation under calibration-label scarcity.