Search papers, labs, and topics across Lattice.
This paper introduces PRISM, a novel framework that adapts subtraction analysis from neuroimaging to analyze the internal mechanisms of large language models (LLMs) by correlating their error profiles with those of post-stroke aphasia patients. By applying this method to both LLMs and human brains, the authors reveal a robust phonemic-favoring dissociation in deep layers of the model and a corresponding cortical cluster, demonstrating that LLMs can exhibit specialized cognitive operations similar to those seen in human language processing. The findings provide a falsifiable approach to testing functional specialization in transformers, paving the way for deeper insights into LLM interpretability and cognitive parallels with human language deficits.
PRISM reveals that large language models exhibit cognitive specialization akin to that of aphasic patients, challenging our understanding of LLM interpretability.
Mechanistic interpretability of large language models lacks spatially resolved, falsifiable tools for testing whether internal components are specialized for distinct cognitive operations. We adapt subtraction analysis, the standard framework of human neuroimaging, from biological brains to perturbed transformers, and apply the same logic to both substrates in parallel. Building on the Brain-LLM Unified Model (BLUM), which showed that layer-perturbed LLaVA-1.6-Vicuna-13B error profiles match the lesion patterns of aphasic patients, we develop PRISM (Perturbation-based Regional Interpretability through Subtraction Mapping). PRISM maps the seven clinical Philadelphia Naming Test categories, subtracts error classes pairwise, and treats each perturbation seed as a subject in a group analysis with threshold-free cluster enhancement along the layer axis. We run a structurally matched analysis on 213 chronic post-stroke aphasia patients using correlation-difference lesion-symptom mapping, and replicate both sides on held-out splits. The designs match in subject dimension (seeds, patients), spatial dimension (layers, atlas-parcellated cortex) and thresholding, but the contrast operator differs: a within-subject error-proportion difference for the LLM, a between-subject correlation difference for the cortex. Both substrates recover a robust phonemic-favoring dissociation, a deep layer cluster and a frontal-perisylvian cortical cluster, both replicating; the semantic-favoring direction is a consistently signed but non-significant trend on both. PRISM thus gives a falsifiable, spatially resolved test of functional-specialization claims in transformer language models. A confirmatory ROI-level intervention (PRISM Stage 3) licensing the strongest causal-mechanism claim is left to subsequent work.
