Search papers, labs, and topics across Lattice.
This paper introduces EviRank, a novel approach to multimodal image re-ranking that treats the task as a semantic constraint satisfaction problem, allowing for precise handling of complex queries. By parsing queries into structured evidence packages with six semantic slots, EviRank effectively captures multifaceted relevance without the pitfalls of traditional embedding methods or free-form reasoning. The method achieves state-of-the-art performance across five benchmarks while enabling a lightweight student model to retain over 90% of the teacher's capabilities at a reduced cost.
EviRank redefines multimodal image re-ranking by transforming complex queries into structured evidence, achieving unprecedented accuracy and efficiency.
Real-world image search queries are multimodal and compositional: ``find this shirt in pink'' specifies an entity to retain, an attribute to modify, and context to ignore. Yet existing re-rankers either compress such multifaceted relevance into an opaque embedding or rely on free-form chain-of-thought that easily omits or hallucinates fine-grained constraints. Drawing on rubric- and checklist-based evaluation from NLP, we recast multimodal image re-ranking as a semantic constraint satisfaction problem and propose EviRank, which parses any query - text-only, image-only, or composed - into a unified evidence package: typed criteria across six semantic slots (e.g., entities, attributes, relations), each labelled required, forbidden, or ignorable. Re-ranking then reduces to evidence-conditioned verification, combining deterministic rubric scoring and evidence-grounded listwise comparison in a single training-free procedure. The explicit evidence can further serve as structured supervision for optionally distilling a lightweight student. Across five benchmarks spanning text-to-image, image-to-image, and composed image retrieval, EviRank achieves state-of-the-art performance, and the distilled student preserves over 90% of the teacher's capability at substantially lower cost.
