Search papers, labs, and topics across Lattice.
This paper investigates hallucinations in small language models by hypothesizing that genuine responses to the same prompt cluster more tightly in the embedding space than hallucinations. They empirically validate this hypothesis and demonstrate a consistent level of separability between genuine and hallucinated responses based on their geometric properties. They then leverage this geometric insight to develop a label-efficient propagation method, requiring only 30-50 annotations, that classifies model responses with high accuracy (F1 > 90%).
Turns out, you can spot LLM hallucinations just by looking at how tightly their answers cluster in embedding space, enabling a surprisingly effective way to flag bad responses with minimal labeling.
Hallucinations -- fluent but factually incorrect responses -- pose a major challenge to the reliability of language models, especially in multi-step or agentic settings. This work investigates hallucinations in small-sized LLMs through a geometric perspective, starting from the hypothesis that when models generate multiple responses to the same prompt, genuine ones exhibit tighter clustering in the embedding space, we prove this hypothesis and, leveraging this geometrical insight, we also show that it is possible to achieve a consistent level of separability. This latter result is used to introduce a label-efficient propagation method that classifies large collections of responses from just 30-50 annotations, achieving F1 scores above 90%. Our findings, framing hallucinations from a geometric perspective in the embedding space, complement traditional knowledge-centric and single-response evaluation paradigms, paving the way for further research.
