Search papers, labs, and topics across Lattice.
This paper investigates the internal mechanisms of Large Language Models (LLMs) by analyzing the dynamic geometry of their hidden state manifolds, revealing that these latent spaces form Small-World networks. The authors demonstrate a significant topological phase transition where early layers remain fragmented, while deeper layers efficiently compress semantic distances, adhering to the "Six Degrees of Separation" principle. Additionally, the study applies this framework to zero-shot hallucination detection in Retrieval-Augmented Generation, showing that factually grounded outputs preserve structural integrity, unlike hallucinations that lead to topological collapse.
Deep LLMs compress vast semantic distances into navigable pathways, revealing a surprising topological phase transition that enhances reasoning capabilities.
Large Language Models (LLMs) demonstrate remarkable multi-hop reasoning capabilities over long contexts, yet the internal mechanisms enabling these distant cognitive leaps remain poorly understood. Traditional attention-based interpretability often fails to capture true semantic proximity due to routing artifacts like attention sinks. In this paper, we bypass attention weights to directly analyze the dynamic geometry of the hidden state manifold, proving that deep LLM latent spaces natively organize into Small-World networks. By sparsifying the continuous similarity matrices of long-context representations into unweighted graphs, we trace the connectivity between highly disjoint semantic anchors across two distinct architectures. Our findings reveal a sharp topological phase transition: while early syntactic layers remain entirely fractured, deep reasoning layers abruptly compress massive conceptual distances into highly navigable pathways strictly bounded by the"Six Degrees of Separation"limit (=<6 semantic hops). Furthermore, we demonstrate the practical efficacy of this framework by applying it to zero-shot hallucination detection within Retrieval-Augmented Generation (RAG) using the RAGognize dataset. We show that factually grounded generations maintain structural integrity with their source context (approximately 3 hops), whereas hallucinations induce severe topological collapse. Ultimately, this work mathematically formalizes how transformers execute abstract reasoning and provides a novel, strictly geometric signature for evaluating factual reliability.
