Search papers, labs, and topics across Lattice.
This paper introduces ArtECulture, a novel benchmark for culture-conditioned visual emotion understanding that addresses the limitations of existing methods by providing 6,792 artworks annotated with culture-specific emotion labels and explanations across English, Chinese, and Arabic cultures. The evaluations of 16 multimodal large language models (MLLMs) show that the task remains challenging, with the best-performing model achieving less than 50% accuracy, highlighting the need for improved approaches. To enhance performance, the authors propose a retrieval-augmented framework that integrates a cultural emotion knowledge base into MLLMs, significantly improving emotion prediction and explanation generation.
Culture-specific emotional perception can be effectively integrated into MLLMs, but current models struggle to achieve even 50% accuracy on this nuanced task.
Existing visual emotion understanding methods typically ignore cultural variations in emotional perception. We introduce culture-conditioned visual emotion understanding, a task that predicts the culture-specific emotional perception of a given image and explains the underlying rationale. Although related benchmarks exist, they are limited by inconsistent individual annotations, which hinder the derivation of majority-supported culture-level emotion labels, and imbalanced cultural coverage. Thus, we present ArtECulture, a benchmark containing 6,792 artworks with culture-specific emotion labels and explanations across English, Chinese, and Arabic cultures, with balanced Western and non-Western content. Evaluations of 16 open- and closed-source Multimodal Large Language Models (MLLMs) under a zero-shot setting reveal that the task remains challenging, with the best model achieving below 50\% accuracy. To address this limitation, we introduce a retrieval-augmented culture-conditioned emotion understanding framework, which leverages a concept-based cultural emotion knowledge base to inject explicit cultural knowledge into MLLMs without additional training. The framework improves both culturally aligned emotion prediction and grounded explanation generation. Our benchmark and code will be publicly released.
