Search papers, labs, and topics across Lattice.
This study introduces Topo-Omni, a topographic multimodal model that integrates visual, auditory, and cognitive processing into a single contiguous in-silico sheet, overcoming the limitations of previous unimodal models. By fine-tuning a pretrained foundation model with a spatial smoothness objective, the model reveals clusters that align with human neuroimaging data and demonstrates that manipulating these clusters can selectively bias perception. The findings not only validate existing neuroimaging insights but also uncover novel clusters related to natural landscapes and animal networks, providing a unified framework for understanding cortical organization.
A single spatial principle can organize multimodal representations, revealing new insights into cortical processing and perception.
Nearby neurons in cortex share similar response profiles, producing systematic spatial organization across sensory and cognitive systems. Recent topographic models reproduce aspects of this structure but remain unimodal and spatially constrain each layer separately, yielding fragmented maps that capture neither the contiguity of cortical processing streams nor their integration across modalities. We introduce Topo-Omni, a topographic multimodal model in which visual, auditory, and language/cognitive processing share a single contiguous in-silico sheet. Built by fine-tuning a pretrained foundation model with a spatial smoothness objective, this architecture develops clusters across modalities that are consistent with human neuroimaging, from sensory to cognitive systems. Driving or suppressing a cluster selectively biases or impairs perception, paralleling human intervention studies. Finally, we use our model to screen for novel clusters in-silico and discover new natural landscape and animal networks which we validate in human data. A single spatial principle thus organizes representations across modalities and processing stages, yielding testable hypotheses about cortical organization.
