Search papers, labs, and topics across Lattice.
This paper investigates the neural plausibility of backpropagation by comparing fMRI and MEG recordings of human brain activity during image processing with backpropagated gradients from pretrained vision models (DINOv3 and others). The study finds that backpropagated gradients can predict brain activity in higher-level visual cortex, but the spatial and temporal organization of these gradients significantly deviates from the hierarchical processing observed in the brain. This suggests that while deep networks and the brain may achieve similar representations, their learning mechanisms are likely fundamentally different.
Backpropagation's gradients, while predictive of high-level visual cortex activity, march to a different hierarchical beat than the brain itself, challenging its status as a biologically plausible learning mechanism.
Backpropagation is the core learning mechanism underlying deep learning. However, whether and how this algorithm is implemented in the brain remains highly debated. In particular, while forward activations of pretrained models reliably map onto the cortical hierarchy of visual processing, it is unknown whether backpropagated gradients exhibit a similar correspondence. Here, we address this question using functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) recordings of human brain responses to natural images. For this, we extend standard encoding analyses of forward activations to map backpropagated gradients onto neural data. Focusing on a recent self-supervised vision model (DINOv3) and reproducing results on eight vision models, we find that backpropagated gradients can reliably predict both fMRI and MEG signals, specifically in higher-level visual cortex and for later latencies. However, the spatial and temporal organization of these backpropagated gradients in the brain diverges from the patterns expected under a biologically plausible backpropagation mechanism: specifically, both the order in which gradients are computed and their spatial organization diverge from the temporal and spatial hierarchies of the human brain. Together, these results suggest that, although deep networks and the brain may share similar representational content, they likely rely on fundamentally different mechanisms to learn those representations.
Meta AI