Search papers, labs, and topics across Lattice.
MARS-CLIP adapts frozen contrastive vision-language representations to zero-shot semantic segmentation by resolving CLIP's spatial resolution bottlenecks and boundary degradation. The framework couples a multi-resolution feature extraction module that blends local fine-grained patches with global context, alongside an attention refinement mechanism that guides the final self-attention layers with intermediate spatial and color priors. Across six public benchmarks, the method achieves state-of-the-art dense prediction performance without requiring mask-level supervision or external decoders.
Zero-shot CLIP can delineate crisp semantic object boundaries without retraining鈥攕imply by routing early-layer spatial and color biases directly into the final self-attention blocks alongside multi-scale tokens.
Contrastive Language-Image Pre-training (CLIP) has demonstrated impressive capabilities in zero-shot transfer but often struggles with dense prediction tasks due to low spatial resolution and the loss of structural information. To address these limitations, we propose MARS-CLIP (Multi-resolution and Attention Refined Segmentation for CLIP), a novel framework for zero-shot semantic segmentation. Our approach introduces two key strategies: (i) a multi-resolution feature extraction module that fuses local fine-grained features with global context to overcome input resolution constraints, and (ii) an attention refinement mechanism that injects spatial and color biases from intermediate layers into the final self-attention block to accurately restore object boundaries. A set of experiments on six public datasets demonstrates that MARS-CLIP significantly outperforms state-of-the-art methods.
