Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study evaluates how well vision models represent color in a manner akin to human perception, using a fuzzy perceptual model with 86 graded categories derived from human survey data. The researchers applied this framework to eleven Vision Transformer encoders, revealing that while category-level results were consistent across models, the graded alignment varied significantly, with Masked Autoencoders (MAE) demonstrating superior alignment. The findings indicate that human-like color grounding encompasses multiple dimensions that cannot be captured by traditional geometric evaluations alone.
Masked Autoencoders outperform other vision models in aligning color representation with human perception, revealing critical nuances in color grounding.
Do vision models see colors the way humans do? Existing evaluations of color representations usually compare them with geometric spaces such as CIELAB or with discrete color labels. These references capture perceptual distance or category membership, but not the graded way in which people organize colors. We evaluate color grounding against a fuzzy perceptual model with 86 graded categories fitted to human survey data. The framework can be applied to any image encoder and measures three complementary properties: category boundaries, category compactness, and graded alignment beyond what color geometry alone can explain. Across eleven Vision Transformer encoders, the category-level results are broadly similar, whereas graded alignment differs substantially. Masked Autoencoders achieve the strongest beyond-geometry alignment, with confidence intervals that do not overlap those of the other encoders. A layer-wise analysis further shows that masked reconstruction preserves this structure toward the output. On natural images, MAE represents surface color globally, while language-supervised models encode color more strongly in relation to the foreground object. These results show that human-like color grounding has several distinct aspects that should not be reduced to a single score.