Search papers, labs, and topics across Lattice.
This paper introduces InsightVQA, a comprehensive dataset designed to enhance visual question answering (VQA) by focusing on emotion understanding and cognitive reasoning. By curating 138K high-confidence images and annotating them across three hierarchical levels鈥攑erception, grounded understanding, and cognition鈥攖he authors provide a robust framework for evaluating models on their ability to reason about emotions and their underlying causes. The results indicate that InsightVQA presents substantial challenges for existing models, highlighting the need for improved approaches in grounded emotion understanding and reasoning tasks.
InsightVQA reveals that current models struggle with high-dimensional emotion-cognitive reasoning, exposing critical gaps in visual understanding capabilities.
Visual emotion understanding requires models not only to recognize emotional states, but also to why they arise and perform higher-level cognitive reasoning. However, existing benchmarks mainly focus on emotion recognition, offering limited support for grounded understanding and response-oriented analysis. To address this gap, we introduce \textbf{InsightVQA}, a large-scale dataset for hierarchical visual question answering on emotion understanding and cognitive reasoning. Building from 351K images collected from six public sources, we apply a rigorous multi-stage filtering pipeline to curate 138K high-confidence images. Each image is annotated at three hierarchical levels: perception QA for emotion and valence recognition, grounded understanding QA constructed from visual trigger extraction through constraint-guided generation, and cognition QA centered on response intent prediction and sequential insight reasoning. In total, InsightVQA contains 725K QA pairs. We further present \textbf{InsightVQA-Bench}, a high-quality evaluation benchmark comprising 30K samples for fine-grained evaluation. To support evaluation, we introduce \textbf{InsightNet}, an emotion-tuned baseline for MLLMs. Results demonstrate that InsightVQA poses significant challenges for grounded emotion understanding and reasoning.
