Search papers, labs, and topics across Lattice.
The paper introduces a geometric taxonomy of LLM hallucinations, categorizing them into three types (center-drift, wrong-well convergence, coverage gaps) based on token embedding cluster structure. It analyzes 11 transformer models, finding universal polarity structure and cluster cohesion, and a significant radial information gradient in most. The study demonstrates how these geometric statistics can be used to detect potential hallucination vulnerabilities and relate them to specific architectural choices in models like ALBERT and MiniLM.
LLM hallucinations aren't random errors, but instead manifest as distinct, geometrically measurable patterns in token embedding spaces, opening the door to targeted detection strategies.
We propose a geometric taxonomy of large language model hallucinations based on observable signatures in token embedding cluster structure. By analyzing the static embedding spaces of 11 transformer models spanning encoder (BERT, RoBERTa, ELECTRA, DeBERTa, ALBERT, MiniLM, DistilBERT) and decoder (GPT-2) architectures, we identify three operationally distinct hallucination types: Type 1 (center-drift) under weak context, Type 2 (wrong-well convergence) to locally coherent but contextually incorrect cluster regions, and Type 3 (coverage gaps) where no cluster structure exists. We introduce three measurable geometric statistics: α (polarity coupling), \b{eta} (cluster cohesion), and λ_s (radial information gradient). Across all 11 models, polarity structure (α > 0.5) is universal (11/11), cluster cohesion (\b{eta} > 0) is universal (11/11), and the radial information gradient is significant (9/11, p < 0.05). We demonstrate that the two models failing λ_s significance -- ALBERT and MiniLM -- do so for architecturally explicable reasons: factorized embedding compression and distillation-induced isotropy, respectively. These findings establish the geometric prerequisites for type-specific hallucination detection and yield testable predictions about architecture-dependent vulnerability profiles.
