Search papers, labs, and topics across Lattice.
The paper introduces MCMR, a new large-scale benchmark for evaluating fine-grained, multi-condition cross-modal retrieval using natural language queries across five product domains. MCMR requires models to jointly satisfy multiple visual and textual attributes specified in the queries, unlike existing benchmarks that focus on coarse-grained or single-condition alignment. Experiments using MLLM-based retrievers and rerankers on MCMR reveal modality asymmetries, the importance of visual cues for early-rank precision, and the effectiveness of pointwise rerankers for fine-grained matching.
Forget simple image search: MCMR reveals how current multimodal models struggle with the complex, interdependent constraints of real-world product search.
Recent advances in multimodal large language models (MLLMs) have substantially expanded the capabilities of multimodal retrieval, enabling systems to align and retrieve information across visual and textual modalities. Yet, existing benchmarks largely focus on coarse-grained or single-condition alignment, overlooking real-world scenarios where user queries specify multiple interdependent constraints across modalities. To bridge this gap, we introduce MCMR (Multi-Conditional Multimodal Retrieval): a large-scale benchmark designed to evaluate fine-grained, multi-condition cross-modal retrieval under natural-language queries. MCMR spans five product domains: upper and bottom clothing, jewelry, shoes, and furniture. It also preserves rich long-form metadata essential for compositional matching. Each query integrates complementary visual and textual attributes, requiring models to jointly satisfy all specified conditions for relevance. We benchmark a diverse suite of MLLM-based multimodal retrievers and vision-language rerankers to assess their condition-aware reasoning abilities. Experimental results reveal: (i) distinct modality asymmetries across models; (ii) visual cues dominate early-rank precision, while textual metadata stabilizes long-tail ordering; and (iii) MLLM-based pointwise rerankers markedly improve fine-grained matching by explicitly verifying query-candidate consistency. Overall, MCMR establishes a challenging and diagnostic benchmark for advancing multimodal retrieval toward compositional, constraint-aware, and interpretable understanding. Our code and dataset is available at https://github.com/EIT-NLP/MCMR
