Search papers, labs, and topics across Lattice.
This paper introduces a self-supervised framework for video object segmentation that combines attention-guided token selection with lightweight temporal clustering to enhance mid-level, part-aware representations. By employing a saliency-weighted symmetric consistency objective, the method effectively maintains spatial accuracy and temporal coherence across frames without relying on extensive labeled datasets. The results demonstrate significant improvements in segmentation performance in unconstrained multi-object scenarios, showcasing the framework's scalability and generalization capabilities.
Achieving accurate video object segmentation without manual labels, this framework outperforms traditional methods by maintaining both spatial and temporal coherence.
Video object segmentation (VOS) is a fundamental task in video understanding, requiring accurate delineation and consistent tracking of objects across frames. While supervised methods achieve strong performance, they rely on densely annotated datasets that are costly to obtain and have limited domain coverage. Self-supervised learning offers a promising alternative by removing the need for manual labels; however, existing approaches often struggle to jointly maintain spatial accuracy and temporal coherence, particularly in unconstrained multi-object scenarios. Many rely on optical flow, synthetic motion cues, or task-specific pretraining, limiting scalability and generalisation. We propose a self-supervised framework, Cross-Temporal Consistency and Clustering, that learns mid-level, part-aware representations by combining attention-guided token selection with lightweight temporal clustering. Instead of operating at the pixel or whole-object level, the method aligns soft part assignments across time using a saliency-weighted symmetric consistency objective. The framework leverages a frozen transformer backbone with lightweight modules for adaptive token selection and multi-offset temporal alignment, enabling efficient scaling across resolutions and motion patterns.
