Search papers, labs, and topics across Lattice.
This paper investigates the emergence of synergistic information processing cores within large language models, drawing parallels to functional differentiation in the human brain. By applying Integrated Information Decomposition, the authors find that middle layers of LLMs exhibit synergistic processing, while early and late layers rely on redundancy. Ablation studies demonstrate that these synergistic components are critical for abstract reasoning, highlighting a potential link between artificial and biological intelligence.
LLMs spontaneously organize into brain-like functional units where the whole is greater than the sum of its parts, and destroying these synergistic cores cripples reasoning.
The evolution of intelligence in artificial systems provides a unique opportunity to identify universal computational principles. Here we show that large language models spontaneously develop synergistic cores where information integration exceeds individual parts remarkably similar to the human brain. Using Integrated Information Decomposition across multiple architectures we find that middle layers exhibit synergistic processing while early and late layers rely on redundancy. This organization is dynamic and emerges as a physical phase transition as task difficulty increases. Crucially ablating synergistic components causes catastrophic performance loss confirming their role as the physical entity of abstract reasoning and bridging artificial and biological intelligence.
