Search papers, labs, and topics across Lattice.
This paper critiques the prevailing use of the Transformer architecture in AI, arguing that it represents a structural error by treating diverse cognitive tasks as solvable by a single model. By drawing parallels with neuroanatomy, it highlights that distinct cognitive functions are best served by qualitatively different structures, as evidenced by the success of convolutional neural networks in image recognition. The authors propose a Heterogeneous Topological Network that emphasizes modularity and tailored architectures, advocating for a design approach that incorporates structural evidence from neuroscience into AI model development.
Treating AI models as a single, scalable architecture may be a fundamental mistake, as distinct cognitive tasks require qualitatively different structures for optimal performance.
AI researchers describe state-of-the-art models as one thing repeated at scale: the Transformer, wired identically for text, pixels, or speech. Neuroscientists describe the cortex as a mosaic - dense Layer 4 in visual cortex for spatial encoding, thick Layers 5/6 in motion cortex for temporal integration - different jobs solved by different structures. This paper argues the gap is a structural error, not a stylistic one, and is measurable. A century of cytoarchitecture, from Brodmann to single-cell Patch-seq, shows distinct cognitive functions are implemented by qualitatively different structures, not by rescaling one template. The convolutional neural network is the field's own proof: local receptive fields and hierarchical depth encoded this prior directly, reaching strong image recognition on far less data than later architectures needed. The paper traces how this lesson was discarded: the"Hardware Lottery"made the Transformer the path of least resistance, not the principled choice, and Mixture-of-Experts, often cited as diversity, in fact partitions parameters among identical experts. A functionalist analysis shows the Transformer is best understood as a functional analog of the hippocampal formation, not a general-purpose cortex - the same mistake as treating cortex as one giant Broca's area, except the field has now standardized on a giant hippocampus, applied to tasks it was never built for: audition, executive gating, working memory. The paper closes with an alternative: a Heterogeneous Topological Network, a System of Systems in which distinct modules keep the inductive bias their computation demands and communicate through standardized interfaces. This is a design discipline for AI architects, not cognitive science: specify modularity before training, using structural evidence as a design input rather than reverse-engineering architecture from a trained model's behavior.
