Search papers, labs, and topics across Lattice.
The paper demonstrates that existing representational similarity metrics are confounded by network scale (depth/width), leading to inflated similarity scores. To address this, they introduce a permutation-based null-calibration framework that provides statistically guaranteed calibrated similarity scores. Applying this framework to the Platonic Representation Hypothesis reveals that global spectral convergence disappears after calibration, while local neighborhood similarity remains significant across modalities.
Neural networks may not converge to a single "Platonic" representation of reality, but they do seem to agree on local neighborhood relationships, suggesting a more nuanced "Aristotelian" view.
The Platonic Representation Hypothesis suggests that representations from neural networks are converging to a common statistical model of reality. We show that the existing metrics used to measure representational similarity are confounded by network scale: increasing model depth or width can systematically inflate representational similarity scores. To correct these effects, we introduce a permutation-based null-calibration framework that transforms any representational similarity metric into a calibrated score with statistical guarantees. We revisit the Platonic Representation Hypothesis with our calibration framework, which reveals a nuanced picture: the apparent convergence reported by global spectral measures largely disappears after calibration, while local neighborhood similarity, but not local distances, retains significant agreement across different modalities. Based on these findings, we propose the Aristotelian Representation Hypothesis: representations in neural networks are converging to shared local neighborhood relationships.
