Search papers, labs, and topics across Lattice.
This paper introduces UMER, a Unified Multimodal Embedding and Ranking framework that enhances universal multimodal retrieval by employing Pair-Aware Discriminative Reasoning to compare query-candidate pairs. By replacing traditional item-wise reasoning with a method that captures instruction-relevant matching and discrepancies, UMER effectively combines contrastive embeddings for global matching with discriminative ranking for nuanced relevance judgments. The framework achieves state-of-the-art performance on the MMEB-V2 benchmark, demonstrating its efficiency and adaptability in corpus-scale retrieval tasks.
Pair-Aware Discriminative Reasoning in UMER reveals critical distinctions between semantically similar candidates, elevating retrieval accuracy in multimodal tasks.
Universal multimodal retrieval aims to support diverse instruction-aware retrieval tasks, demanding both efficient corpus-scale matching and fine-grained semantic reasoning. Recent MLLM-based embedding methods typically derive representations from hidden states, while Chain-of-Thought (CoT) reasoning is emerging as a promising strategy for embedding enhancement by encoding intermediate semantic evidence into the representation space. However, existing CoT methods typically use item-wise reasoning over queries and candidates in isolation, providing no explicit evidence to distinguish a positive from a semantically confusable hard negative. Moreover, contrastive embeddings capture global similarity but struggle with meta-tasks requiring answer verification, category judgment or fine-grained reasoning. In this paper, we propose UMER, a Unified Multimodal Embedding and Ranking framework for universal multimodal retrieval. UMER replaces item-wise reflection with Pair-Aware Discriminative Reasoning, which compares query--candidate pairs to identify instruction-relevant matching and discrepancy evidence. UMER jointly learns contrastive embeddings for efficient global matching and discriminative ranking for explicit pairwise relevance judgment within a single MLLM. A complementary mutual distillation strategy further transfers reliable pairwise preferences between the embedding and ranking functions. On the MMEB-V2 benchmark, UMER achieves state-of-the-art performance under comparable experimental settings while supporting budget-adjustable inference.
