Search papers, labs, and topics across Lattice.
This paper introduces DINOde, an ODE-based framework that enhances open-vocabulary semantic segmentation (OVSS) by continuously aligning CLIP text embeddings with the DINO visual manifold. By employing Semantic Text Flow and Global Context Flow, DINOde effectively evolves text embeddings and refines image representations while preserving the hyperspherical geometry of the feature space through Velocity Tangent Projection. Extensive experiments reveal that DINOde outperforms existing methods, achieving state-of-the-art results across multiple OVSS benchmarks, thus addressing the limitations of previous approaches that relied on discrete MLP projections.
Continuous alignment of text and visual representations in DINOde leads to significant performance gains in open-vocabulary semantic segmentation, outperforming traditional methods.
Open-vocabulary semantic segmentation (OVSS) leverages textual semantics to segment objects beyond predefined categories. While the self-supervised model DINOv3 provides strong structured visual representations, its lack of native textual alignment hinders its direct application to OVSS. To bridge this gap, we propose DINOde, an ODE-based framework that continuously aligns CLIP text embeddings with the DINO visual manifold. Our approach employs two complementary components: (i) Semantic Text Flow (STF), which evolves text embeddings toward the DINO manifold through a continuous ODE trajectory, and (ii) Global Context Flow (GCF), which progressively refines the holistic image representation carried by DINO's CLS token. To preserve the hyperspherical geometry of the feature space during this evolution, we further introduce Velocity Tangent Projection, which constrains the learned velocity field to the tangent space. By modeling alignment as a continuous trajectory, DINOde avoids the manifold entanglement inherent in discrete MLP projections and yields more robust cross-modal alignment. Extensive experiments demonstrate that DINOde consistently outperforms existing methods and achieves state-of-the-art performance across multiple OVSS benchmarks. The code is available at https://github.com/yoon307/DINOde.
