Search papers, labs, and topics across Lattice.
This paper introduces Text-to-Seed (T2S), a training-free framework for open-vocabulary semantic segmentation that utilizes Stable Diffusion to generate accurate seed points based on text queries. By reformulating the segmentation process to focus on text-guided seed localization followed by region expansion, T2S significantly improves segmentation reliability compared to traditional methods that rely on coarse masks. The results show that T2S outperforms existing OVSS benchmarks without requiring task-specific training or additional annotations, highlighting the effectiveness of combining semantic grounding with seed-driven segmentation.
T2S transforms open-vocabulary semantic segmentation by generating precise seed points from text, leading to superior segmentation without the need for training.
Open-vocabulary semantic segmentation (OVSS) aims to segment image regions corresponding to arbitrary text queries. Although the Segment Anything Model (SAM) is a powerful foundation model for segmentation, its standalone performance on OVSS remains limited. Existing methods therefore often use SAM to refine coarse masks predicted by other models, but this strategy is unreliable when the initial masks are inaccurate. In this work, we argue that more reliable segmentation can be achieved by exploiting SAM as a region expansion module guided by accurate object points (i.e., seeds) rather than inaccurate coarse masks. Inspired by classical seeded segmentation, we reformulate OVSS as text-guided seed localization followed by seed-based region expansion. To realize this idea, we propose Text-to-Seed (T2S), a training-free framework that leverages the text-to-region correspondence of Stable Diffusion to generate attention-based seed points for target categories described by text. These sparse seeds are then used as point prompts for SAM to produce full object masks. Without task-specific training or additional annotations, T2S achieves strong performance on standard OVSS benchmarks, demonstrating the effectiveness of combining semantic grounding with seed-driven spatial segmentation.
