Search papers, labs, and topics across Lattice.
This paper introduces Dual-State Slot Attention (DSSA), a self-supervised framework that effectively decouples object appearance from identity in video object-centric learning. By addressing the inherent conflicts in existing slot-based methods, DSSA employs a local state for per-frame appearance and an identity state for stable object representation, which enhances both segmentation quality and temporal consistency. Experimental results on multiple datasets, including MOVi-C and YouTube-VIS, show that DSSA outperforms prior methods, leading to improved downstream tasks such as object recognition and video dynamics prediction.
Separating appearance from identity in video object learning leads to significant gains in segmentation and temporal consistency, outperforming existing methods.
Unsupervised video object-centric learning aims to decompose dynamic scenes into persistent, object-level representations without supervision. However, existing slot-based methods struggle to maintain stable object identity in challenging settings such as rapid motion and partial occlusion. First, they typically encode both the per-frame appearance of an object and its identity across frames in a single slot vector, creating an objective conflict that leads to slot swapping: reconstruction requires sensitivity to transient visual changes, whereas temporal consistency requires invariance to them. Second, the token renormalization used in Slot Attention can amplify weakly attending slots, allowing them to absorb tokens from other objects and destabilize slot-to-object correspondence. We propose Dual-State Slot Attention (DSSA), a fully self-supervised framework that addresses these limitations by separating appearance from identity and by reducing spurious updates from weakly matching slots. DSSA decomposes each slot into a local state for per-frame appearance and an identity state for temporally stable object information, thereby aligning reconstruction and temporal consistency with separate representations. The identity state is updated through a learned recurrent transition that acts as a temporal filter on the local state, while competition-modulated aggregation (CMA) down-weights updates from weakly matching slots and prevents them from absorbing tokens from other objects. Experiments on MOVi-C, MOVi-D, and YouTube-VIS demonstrate that DSSA consistently improves segmentation quality and temporal consistency over prior methods, while also yielding stronger downstream object recognition and video dynamics prediction. Code and models will be made publicly available upon acceptance.
