Search papers, labs, and topics across Lattice.
This paper introduces EOVSAM, an efficient open-vocabulary segmentation framework that modifies SAM 3 for single-pass predictions, eliminating the need for prompt conditioning. By implementing a novel Attentional Aggregation strategy, EOVSAM enhances segmentation accuracy while significantly reducing computational overhead, achieving inference speed improvements of up to 338 times compared to vanilla SAM 3. Experimental results demonstrate that EOVSAM not only outperforms existing methods in segmentation accuracy across various datasets but also excels in maintaining performance at lower resolutions.
Achieving up to 338x faster inference, EOVSAM redefines open-vocabulary segmentation without sacrificing accuracy.
Open-vocabulary segmentation identifies and segments objects from arbitrary textual descriptions. SAM 3 supports noun-phrase-guided segmentation and achieves competitive open-vocabulary performance through exhaustive vocabulary traversal, yet suffers from prohibitive computational overhead as target categories scale. In this paper, we propose an Efficient Open-Vocabulary segmentation framework with SAM 3 (EOVSAM), which adapts SAM 3 for single-pass prediction. EOVSAM removes prompt conditioning to turn SAM 3 into an efficient mask generator and introduces a new Attentional Aggregation strategy to optimize open-vocabulary classification end-to-end. This formulation avoids the multi-stage pipelines and post-processing heuristics commonly used by existing methods, while mitigating the closed-set collapse that can arise when classification is optimized directly. EOVSAM consistently improves segmentation accuracy over vanilla SAM 3 on all evaluated datasets and accelerates inference by up to 338$\times$. Furthermore, EOVSAM maintains high accuracy at lower resolutions while achieving even more remarkable inference speeds. Experiments on standard semantic and panoptic segmentation benchmarks show that EOVSAM combines competitive or state-of-the-art accuracy with a substantial speed advantage over existing open-vocabulary segmentation models. Code and models are available at https://github.com/hustvl/EOVSAM.
