Search papers, labs, and topics across Lattice.
This paper introduces DIM-Fashion, a benchmark comprising 26,000 multi-turn sessions derived from 13 fashion retrieval datasets, designed to capture diverse intent transitions and rollback behaviors in interactive fashion search. The authors propose FashionAM, a multimodal language model and vision-language pretraining framework that directly aligns multimodal conversational queries with a fashion-oriented gallery embedding space, circumventing the limitations of textification. Experimental results show that FashionAM significantly outperforms existing multi-turn retrieval methods, highlighting its effectiveness in preserving fine-grained visual cues during the retrieval process.
FashionAM revolutionizes fashion image retrieval by directly linking multimodal queries to visual embeddings, eliminating the pitfalls of textification.
Real-world fashion search involves interactive retrieval across multiple turns. However, existing multi-turn retrieval methods are built on a restrictive assumption that every interaction follows the same attribute-editing paradigm, leaving heterogeneous intent transitions unexplored. Moreover, existing approaches often rely on textification to bridge multimodal queries and visual retrieval, which may lose fine-grained visual cues. To address these gaps, we introduce DIM-Fashion, a benchmark of 26K multi-turn sessions constructed from 13 fashion retrieval datasets across 7 tasks, featuring diverse intent transitions and rollback behaviors. We further propose FashionAM, an MLLM-VLP framework that directly aligns multimodal conversational queries with a fashion-oriented gallery embedding space, avoiding intermediate textification. Extensive experiments demonstrate the effectiveness of FashionAM over existing approaches. The dataset and code will be made publicly available upon acceptance.
