Search papers, labs, and topics across Lattice.
ModuSeg, a novel training-free framework for weakly supervised semantic segmentation, decouples object discovery (via a mask proposer) from semantic assignment (via feature retrieval from a semantic foundation model). This decoupling avoids the typical entanglement of semantic recognition and object localization that leads to models focusing on sparse, discriminative regions. By integrating semantic boundary purification and soft-masked feature aggregation, ModuSeg mitigates boundary ambiguity and quantization errors, leading to high-quality category prototypes and improved segmentation performance.
Achieve surprisingly strong weakly supervised segmentation without any training by decoupling object proposals from semantic feature retrieval using foundation models.
Weakly supervised semantic segmentation aims to achieve pixel-level predictions using image-level labels. Existing methods typically entangle semantic recognition and object localization, which often leads models to focus exclusively on sparse discriminative regions. Although foundation models show immense potential, many approaches still follow the tightly coupled optimization paradigm, struggling to effectively alleviate pseudo-label noise and often relying on time-consuming multi-stage retraining or unstable end-to-end joint optimization. To address the above challenges, we present ModuSeg, a training-free weakly supervised semantic segmentation framework centered on explicitly decoupling object discovery and semantic assignment. Specifically, we integrate a general mask proposer to extract geometric proposals with reliable boundaries, while leveraging semantic foundation models to construct an offline feature bank, transforming segmentation into a non-parametric feature retrieval process. Furthermore, we propose semantic boundary purification and soft-masked feature aggregation strategies to effectively mitigate boundary ambiguity and quantization errors, thereby extracting high-quality category prototypes. Extensive experiments demonstrate that the proposed decoupled architecture better preserves fine boundaries without parameter fine-tuning and achieves highly competitive performance on standard benchmark datasets. Code is available at https://github.com/Autumnair007/ModuSeg.
