Search papers, labs, and topics across Lattice.
The paper introduces DailyClue, a new benchmark designed to evaluate the visual clue-driven reasoning capabilities of MLLMs in realistic daily scenarios. The benchmark consists of a dataset spanning four domains and 16 subtasks, requiring MLLMs to actively identify and leverage visual clues for reasoning, rather than relying on pre-existing knowledge or simple perception. Evaluation of various MLLMs and agentic models on DailyClue reveals that accurate identification of visual clues remains a significant challenge for current models.
MLLMs still struggle to reason about everyday situations when they require identifying and using visual clues, despite excelling at tasks relying on pre-existing knowledge.
Daily scenarios are characterized by visual richness, requiring Multimodal Large Language Models (MLLMs) to filter noise and identify decisive visual clues for accurate reasoning. Yet, current benchmarks predominantly aim at evaluating MLLMs' pre-existing knowledge or perceptual understanding, often neglecting the critical capability of reasoning. To bridge this gap, we introduce DailyClue, a benchmark designed for visual clue-driven reasoning in daily scenarios. Our construction is guided by two core principles: (1) strict grounding in authentic daily activities, and (2) challenging query design that necessitates more than surface-level perception. Instead of simple recognition, our questions compel MLLMs to actively explore suitable visual clues and leverage them for subsequent reasoning. To this end, we curate a comprehensive dataset spanning four major daily domains and 16 distinct subtasks. Comprehensive evaluation across MLLMs and agentic models underscores the formidable challenge posed by our benchmark. Our analysis reveals several critical insights, emphasizing that the accurate identification of visual clues is essential for robust reasoning.
Tsinghua AI