Search papers, labs, and topics across Lattice.
This paper introduces MASCOT, a novel approach to enhance Result Diversification (RD) in text-to-image retrieval by framing multi-attribute diversity as a resource allocation problem rather than relying on manifold-based repulsion methods like MS-DPP. MASCOT achieves a significant improvement in early-rank recall, maintaining 88.58% at R@10 compared to MS-DPP's 67.63%, particularly excelling under composite constraints where both temporal and geographic diversity are required. The findings highlight MASCOT's ability to effectively balance relevance and diversity, addressing a critical limitation in existing retrieval systems.
MASCOT outperforms traditional methods by preserving early-rank recall under complex diversity constraints, achieving R@10 of 94.10% while MS-DPP collapses under similar conditions.
Vision-Language Models (VLMs) are highly effective in retrieving semantically relevant images. However, in practice, relevance alone is often insufficient. Systems must also achieve Result Diversification (RD) across composite attributes such as geography and time, a task for which precise control remains challenging. Current re-ranking methods, such as Multi-Source Determinantal Point Processes (MS-DPP), address this using manifold-based repulsion over similarity representations. Although this strategy is effective for broad exploration, it exposes a key limitation in manifold-based models: when subjected to diversity-decrease tasks on discrete metadata, they suffer substantial degradation in early-rank recall. To bridge this gap, we introduce MASCOT (Model-Aware Submodular Coverage for Composite-Attribute Text-to-Image Retrieval). Instead of relying on manifold repulsion, MASCOT formulates multi-attribute diversity as a resource allocation problem, projecting attributes into a soft-binning space weighted by query-driven importance. Averaged across the three PixelProse diversity-decrease tasks, MASCOT preserves an early-rank recall (R@10) of 88.58%, while MS-DPP retains 67.63%. The margin widens under composite constraints: on PP_geo_hour, where temporal and geographic diversity must be suppressed simultaneously, MS-DPP's recall collapses from 0.9737 to 0.4931 and its top-ranked result degrades to R@1 = 0.23, while MASCOT holds R@10 = 0.9410 and R@1 = 0.7202 at a diversity metric above the unconstrained baseline. We do not claim uniform superiority: on aggregate diversity-relevance scores our own simpler ablations attain higher harmonic means on all three decrease tasks, and MASCOT's advantage is specific to recall beyond rank 1 under composite constraints.
