Search papers, labs, and topics across Lattice.
This paper introduces A-MAR, an agent-based framework for multimodal art retrieval that uses structured reasoning plans to guide evidence retrieval for artwork understanding. A-MAR decomposes user queries into multi-step reasoning plans, enabling targeted evidence selection and grounded explanations. Evaluations on SemArt, Artpedia, and a new benchmark ArtCoT-QA demonstrate that A-MAR outperforms static retrieval and strong MLLM baselines in explanation quality, evidence grounding, and multi-step reasoning.
Forget relying on implicit reasoning: A-MAR's explicit reasoning plans unlock better artwork understanding by strategically retrieving relevant evidence.
Understanding artworks requires multi-step reasoning over visual content and cultural, historical, and stylistic context. While recent multimodal large language models show promise in artwork explanation, they rely on implicit reasoning and internalized knowl- edge, limiting interpretability and explicit evidence grounding. We propose A-MAR, an Agent-based Multimodal Art Retrieval framework that explicitly conditions retrieval on structured reasoning plans. Given an artwork and a user query, A-MAR first decomposes the task into a structured reasoning plan that specifies the goals and evidence requirements for each step. Retrieval is then conditionedon this plan, enabling targeted evidence selection and supporting step-wise, grounded explanations. To evaluate agent-based multi- modal reasoning within the art domain, we introduce ArtCoT-QA. This diagnostic benchmark features multi-step reasoning chains for diverse art-related queries, enabling a granular analysis that extends beyond simple final answer accuracy. Experiments on SemArt and Artpedia show that A-MAR consistently outperforms static, non planned retrieval and strong MLLM baselines in final explanation quality, while evaluations on ArtCoT-QA further demonstrate its advantages in evidence grounding and multi-step reasoning ability. These results highlight the importance of reasoning-conditioned retrieval for knowledge-intensive multimodal understanding and position A-MAR as a step toward interpretable, goal-driven AI systems, with particular relevance to cultural industries. The code and data are available at: https://github.com/ShuaiWang97/A-MAR.
