Search papers, labs, and topics across Lattice.
This paper introduces DiCE-CIR, a novel direct composition learning approach for zero-shot composed image retrieval (ZS-CIR) that eliminates the need for projection-based methods, thereby enhancing efficiency and reducing training complexity. By leveraging large-scale image-caption pairs and a large language model to automatically generate compositional training samples, the method trains a lightweight composition module that aligns closely with target representations and ensures edit-consistent transformations. Experimental results demonstrate that DiCE-CIR achieves state-of-the-art performance on the CIRCO benchmark and competitive results on CIRR, all while maintaining high computational efficiency.
Direct composition learning in DiCE-CIR achieves state-of-the-art zero-shot image retrieval without the overhead of projection methods, streamlining the process significantly.
Zero-shot composed image retrieval (ZS-CIR) aims to retrieve a target image from a multimodal query consisting of a reference image and an edit text describing the desired modification. Recent ZS-CIR studies have relied on projection-based methods that map a reference image into pseudo-word tokens in the text embedding space. However, such methods require additional projection and re-encoding steps, increasing training complexity, reducing efficiency, and introducing a discrepancy between training and inference. In this paper, we propose DiCE-CIR, a direct composition learning method that predicts composed query representations by directly composing a reference image and an edit text. To enable scalable training without manually annotated triplets, we automatically construct compositional training samples from large-scale image-caption pairs using a large language model. Based on these samples, we train a lightweight composition module with objectives that promote alignment with the target, edit-consistent semantic transformation, and retrieval discriminability. We conduct extensive experiments on ZS-CIR benchmarks and show that DiCE-CIR achieves state-of-the-art performance on CIRCO and competitive performance on CIRR while maintaining high computational efficiency.
