Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study benchmarks the Llama 3.2 model in the context of semantic fMRI neural language decoding, enhancing the Huth encoding model by expanding voxel selection and utilizing a more advanced proposal model, resulting in an 11% relative improvement in METEOR scores. Additionally, the introduction of fMRIFlamingo, which integrates BOLD activity with Llama-3.2, achieves a notable 42.86% Top-1 accuracy, although a control ablation indicates that this success is largely attributable to the language model's prior rather than the fMRI data itself. These findings highlight the limitations of high-capacity language models in improving fMRI decoding and underscore the necessity for rigorous evaluation methods to discern genuine decoding capabilities from model artifacts.
High-capacity language models can obscure failures in fMRI decoding, revealing that apparent successes may not stem from neural input.
Decoding continuous language from fMRI signals remains a core challenge in non-invasive brain-computer interface research. We present two complementary investigations. First, we improve the Huth et al. ridge regression encoding pipeline through expanded voxel selection (10K->15K), substitution of GPT-2 medium for GPT-1 as the beam-search proposal model, and GPU-accelerated bootstrap training, achieving mean METEOR = 0.149 and BLEU-1 = 0.200 across three held-out narratives for subject UTS03 -- an 11% relative METEOR gain over our replication baseline. Second, we introduce fMRIFlamingo, which maps BOLD activity to a frozen Llama-3.2-1B with trainable gated cross-attention layers via a learned brain tokenizer and a Perceiver Resampler. Despite achieving 42.86% Top-1 accuracy on a 1-in-100 ranking task, well above chance, a blind control ablation with zeroed fMRI inputs yields near-identical scores, revealing that apparent decoding success is driven primarily by the frozen language prior rather than by neural input. These results demonstrate that high-capacity language models do not inherently improve fMRI decoding and can actively obscure failures without rigorous blind-control evaluation.