Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces CLAY, a novel method for adaptive image retrieval that leverages the embedding space of pre-trained Vision-Language Models (VLMs) to enable text-conditional similarity computation without further training. CLAY decouples textual conditioning from visual feature extraction, allowing for efficient multi-conditioned retrieval using fixed visual embeddings. Experiments on standard datasets and a new synthetic dataset, CLAY-EVAL, demonstrate that CLAY achieves high retrieval accuracy with improved computational efficiency compared to existing approaches.
Unlock flexible image retrieval: CLAY lets you reshape your VLM's visual similarity space on the fly with text prompts, no retraining needed.
Human perception of visual similarity is inherently adaptive and subjective, depending on the users' interests and focus. However, most image retrieval systems fail to reflect this flexibility, relying on a fixed, monolithic metric that cannot incorporate multiple conditions simultaneously. To address this, we propose CLAY, an adaptive similarity computation method that reframes the embedding space of pretrained Vision-Language Models (VLMs) as a text-conditional similarity space without additional training. This design separates the textual conditioning process and visual feature extraction, allowing highly efficient and multi-conditioned retrieval with fixed visual embeddings. We also construct a synthetic evaluation dataset CLAY-EVAL, for comprehensive assessment under diverse conditioned retrieval settings. Experiments on standard datasets and our proposed dataset show that CLAY achieves high retrieval accuracy and notable computational efficiency compared to previous works.