Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the internal geometric structure of color representations in multilingual language models, decomposing them into perceptual alignment and cross-lingual symmetry axes. Analysis reveals that perceptual grounding is surprisingly weak and transient, peaking in intermediate layers before collapsing, while cross-lingual symmetry degrades with depth. Crucially, the degree of perceptual alignment in intermediate layers strongly correlates with downstream color classification performance, indicating a dissociation between learned color-language geometry and universal color-perception geometry.
Multilingual LMs encode a universal color-language geometry, but not a universal color-perception geometry, with perceptual grounding fading in later layers.
Do multilingual language models learn a universal internal geometry for color, and is it perceptually grounded or primarily linguistic? We decompose color representations into two geometric axes: (P1) alignment to perceptual color space, and (P2) cross-lingual symmetry. Analyzing layerwise embeddings across models and multilingual datasets, we find that perceptual grounding is weak: alignment to CIE-LAB peaks in intermediate layers, then collapses toward output layers. Cross-lingual symmetry degrades with depth, suggesting later layers encode language-specific structure. Functionally, we show that geometric properties predict task performance: models achieve 96.7% accuracy on color classification at layers where P1 alignment peaks, with P1 alignment (r=0.70) significantly correlating with downstream capabilities. Our results reveal a dissociation: text-only LMs learn universal color-language geometry, not universal color-perception geometry, providing diagnostics for where perceptual structure is preserved in language models.