Search papers, labs, and topics across Lattice.
This paper introduces SynCLIP, a framework designed to enhance open-vocabulary dense perception (OVDP) by addressing synonym-induced grounding inconsistencies that plague existing CLIP-based methods. By implementing a Semantic-consistent Spatial Attention alignment (SSA) module and a Spatial Attention Refinement (SAR) module, SynCLIP minimizes attention map discrepancies and strengthens semantically relevant regions, leading to improved robustness in real-world applications. Experimental results demonstrate that SynCLIP achieves state-of-the-art performance in grounding consistency across various linguistic variants, significantly outperforming previous approaches.
SynCLIP resolves synonym-induced grounding inconsistencies, achieving unprecedented robustness in open-vocabulary dense perception tasks.
Open-vocabulary dense perception (OVDP) aims to localize objects unseen during training by leveraging textual knowledge. Despite the remarkable progress of recent CLIP-based approaches, we identify a critical limitation: synonym-induced grounding inconsistency, where semantically equivalent expressions yield disparate spatial attention patterns. This inconsistency undermines the robustness and performance of existing methods in real-world OVDP applications. To address this issue, we propose SynCLIP, a Synonym-Coherent Language-Image Pretraining framework that enhances synonym-robust grounding for OVDP. SynCLIP introduces a Semantic-consistent Spatial Attention alignment (SSA) module to enhance spatial attention consistency by minimizing discrepancies between attention maps of original and synonymous expressions. Furthermore, a Spatial Attention Refinement (SAR) module selectively strengthens the most semantically relevant spatial regions within aligned maps for more precise and stable grounding. To support synonym-coherent pretraining, we also construct a Synonym-Enriched Visual Corpus (SEViC), which augments each category with multiple synonyms and textual definitions. Extensive experiments on multiple benchmarks demonstrate that SynCLIP substantially improves grounding consistency under diverse linguistic variants and achieves state-of-the-art performance among CLIP-based OVDP methods. Code is available at https://github.com/Justlovesmile/SynCLIP.
