Search papers, labs, and topics across Lattice.
This paper introduces FlowCIR, a novel approach to zero-shot composed image retrieval (ZS-CIR) that utilizes conditional flow matching to facilitate semantic transport between reference and target embeddings. By employing a lightweight transport field, FlowCIR significantly reduces the training resources required鈥攁pproximately ten times less than previous textual-inversion methods鈥攚hile enhancing robustness against common failure modes such as negation. Extensive experiments confirm that FlowCIR achieves competitive performance on standard CIR benchmarks, marking a significant advancement in the efficiency and effectiveness of ZS-CIR techniques.
FlowCIR slashes training resource requirements by 90% while boosting robustness against negation in zero-shot image retrieval tasks.
Zero-shot composed image retrieval (ZS-CIR) aims to retrieve a target image by editing a reference image with a natural-language instruction, without relying on domain-specific annotated triplets. Most existing ZS-CIR methods rely on textual inversion to translate the reference image into pseudo-text tokens and then compose them with the instruction via simple concatenation in the text space, which can be lossy and brittle for fine-grained semantics. In this work, we propose a new paradigm, namely FlowCIR, that casts ZS-CIR as conditional semantic transport between reference and target embeddings. Leveraging \emph{conditional flow matching}, our model learns a lightweight transport field that maps the instruction representation toward a target-aligned query embedding conditioned on the reference image. Since FlowCIR operates on pre-extracted VLM embeddings and trains only a small transport module without updating the image or text encoder, it offers a computationally efficient training protocol compared with prior textual-inversion-based approaches. The resulting framework is training-efficient, requiring roughly $10\times$ fewer training resources than prior textual-inversion-based approaches. We further identify negation and removal as a major failure mode of VLM-based composition. To address this, we propose an inference-only Multi-Negative Steering strategy that steers a negation-containing relative instruction away from its negated semantics, mitigating the limited negation handling of VLMs and improving robustness on negation-heavy queries. Extensive experiments on standard CIR benchmarks demonstrate that FlowCIR achieves strong and competitive performance compared with recent ZS-CIR methods.
