Search papers, labs, and topics across Lattice.
This paper introduces ZooClaw-FashionSigLIP2, a fashion-specialized model that effectively balances the tradeoff between performance on a specific retrieval task and the generalization capabilities of a foundation vision-language encoder. By employing full fine-tuning with knowledge distillation on curated in-domain data, followed by weight interpolation with the base model, it significantly outperforms existing methods, including LoRA and larger models, across all evaluated benchmarks. Additionally, the authors provide a new high-quality fashion retrieval benchmark and a comprehensive analysis of existing benchmarks to address structural biases, enhancing the robustness of future research in this domain.
Outperforming larger models and existing methods, ZooClaw-FashionSigLIP2 sets a new standard for fashion retrieval while addressing critical biases in benchmark evaluations.
Adapting a foundation vision-language encoder to a specialized retrieval task creates a fundamental tradeoff: gains on the target distribution come at the cost of the foundation model's broad generalization, and fashion retrieval is a stringent instance of this problem. We present ZooClaw-FashionSigLIP2, a fashion-specialized SigLIP2-base model that resolves this tradeoff with a simple recipe -- full fine-tuning with knowledge distillation on curated in-domain data, followed by \wiseft~\citep{wortsman2022wiseft} weight interpolation with the base model -- and outperforms LoRA, larger backbones (up to 1B parameters), and external training data. Under fair evaluation, ZooClaw-FashionSigLIP2 outperforms all baselines on every benchmark in our suite. In addition, we release ZooClaw-Fashion, a new high-quality fashion retrieval benchmark, and a systematic quality analysis of widely-used benchmarks that exposes and mitigates structural biases in their public ground truth. We open-source the model weights and all evaluation artifacts to facilitate future research.
