Search papers, labs, and topics across Lattice.
RABBiT is a novel foundation model designed to predict brain responses to speech by leveraging a compact audio-to-fMRI encoder that adapts rapidly to individual participants. It achieves state-of-the-art performance in zero-shot prediction of fMRI responses across diverse auditory and language-selective regions, outperforming traditional models that rely on group averages. By utilizing learned region-specific attention and decomposing brain responses into shared and individual components, RABBiT requires minimal participant-specific data for effective tuning, thus facilitating scalable analyses of language processing in the brain.
RABBiT can accurately predict brain responses to speech with just 10 minutes of participant data, outperforming traditional models and enabling scalable population-level studies.
Language understanding in the brain is context-dependent, varying across experimental stimuli and individuals, which makes it difficult to build computational models that generalize across both. This calls for a foundation model of language-evoked brain activity that can capture shared structure while adapting efficiently to new participants and inputs. We introduce RABBiT (Rapidly Adaptive BOLD foundation model via BraIn-Tuning), a compact audio-to-fMRI encoder designed for accurate zero- and few-shot prediction. A comprehensive evaluation on 324 participants across multiple unseen fMRI datasets shows that RABBiT enables accurate zero-shot prediction of fMRI responses to natural speech across auditory and language-selective regions, surpassing the SOTA foundation model for fMRI and predictions based on group averages. With as little as 10 minutes of participant-specific data, RABBiT further improves performance via parameter-efficient tuning, substantially outperforming per-participant linear models. RABBiT's performance is driven by two key innovations: (1) learned region-specific attention, and (2) a decomposition of brain responses into shared and subject-specific components, combined with a brain-tuned speech backbone. In addition to supporting strong predictive accuracy, the structured, region-specific representations that RABBiT learns enable interpretability. By eliminating the need for extensive per-participant data and model fitting, RABBiT enables scalable population-level analyses of language in the human brain. We make the code available at https://github.com/bridge-ai-neuro/rabbit.
CMU ML