Search papers, labs, and topics across Lattice.
This paper introduces VOS-Agent, a novel collaborative framework for complex video object segmentation that enhances target propagation in challenging scenarios such as occlusion and identity shifts. By integrating SAM3 as a dense segmentation module and employing specialized agents tailored to target characteristics, VOS-Agent effectively addresses the limitations of uniform inference paths, particularly for tiny and semantic-dominated targets. The approach achieved a top score of 69.82% on the $\mathcal{J}\&\dot{\mathcal{F}}$ metric, securing first place in the MOSEv2 Track of the 8th LSVOS Challenge at ECCV 2026.
VOS-Agent outperforms existing methods by leveraging specialized agents for different target types, achieving state-of-the-art results in video object segmentation.
Complex video object segmentation requires robust target propagation under severe occlusion, disappearance and reappearance. Although SAM3 provides strong promptable mask propagation, a uniform inference path remains unreliable for tiny targets with insufficient visual evidence and semantic-dominated targets whose identities depend on explicit attributes. To this end, we present VOS-Agent, a collaborative framework that retains SAM3 as the shared dense segmentation module and conditionally activates specialized agents according to target characteristics. A Target Perception and Routing Agent assigns each sequence to a regular, tiny, or semantic-dominated route. Tiny targets are supported by a Visual Tracking Agent through confidence-aware box prompts, while semantic-dominated targets are handled by an MLLM-based Semantic Agent through description-guided localization and candidate verification. On the MOSEv2 test set, VOS-Agent achieves 69.82% on the official $\mathcal{J}\&\dot{\mathcal{F}}$ metric and ranks first in the MOSEv2 Track of the 8th LSVOS Challenge at ECCV 2026.
