Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces G-MIXER, a training-free zero-shot composed image retrieval (ZS-CIR) method that addresses the limitations of relying solely on explicit textual semantics generated by MLLMs. G-MIXER constructs composed query features by applying geodesic mixup to reference image-text pairs, thereby capturing implicit semantics and generating a diverse candidate set. These candidates are then re-ranked using explicit semantics from MLLMs, leading to state-of-the-art performance on multiple ZS-CIR benchmarks.
Zero-shot image retrieval gets a boost: G-MIXER mixes up image-text features along geodesics to capture implicit semantics, outperforming methods that rely solely on MLLM-generated text.
Composed Image Retrieval (CIR) aims to retrieve target images by integrating a reference image with a corresponding modification text. CIR requires jointly considering the explicit semantics specified in the query and the implicit semantics embedded within its bi-modal composition. Recent training-free Zero-Shot CIR (ZS-CIR) methods leverage Multimodal Large Language Models (MLLMs) to generate detailed target descriptions, converting the implicit information into explicit textual expressions. However, these methods rely heavily on the textual modality and fail to capture the fuzzy retrieval nature that requires considering diverse combinations of candidates. This leads to reduced diversity and accuracy in retrieval results. To address this limitation, we propose a novel training-free method, Geodesic Mixup-based Implicit semantic eXpansion and Explicit semantic Re-ranking for ZS-CIR (G-MIXER). G-MIXER constructs composed query features that reflect the implicit semantics of reference image-text pairs through geodesic mixup over a range of mixup ratios, and builds a diverse candidate set. The generated candidates are then re-ranked using explicit semantics derived from MLLMs, improving both retrieval diversity and accuracy. Our proposed G-MIXER achieves state-of-the-art performance across multiple ZS-CIR benchmarks, effectively handling both implicit and explicit semantics without additional training. Our code will be available at https://github.com/maya0395/gmixer.