Search papers, labs, and topics across Lattice.
This paper introduces SAM2Dual, a novel enhancement for long-term video object segmentation that operates without the need for model retraining. By implementing a Dual Memory architecture that distinguishes between short-term and long-term memory, the method effectively mitigates drift caused by unreliable predictions during extended occlusions and scene changes. The addition of Text-Aware Memory (TAM) further refines memory contributions based on semantic cues, resulting in significant performance improvements across long-term benchmarks, with J&F scores rising from 49.33 to 50.65 on MOSEv2.
Long-term video segmentation just got a boost鈥擲AM2Dual raises performance by leveraging a novel Dual Memory approach that adapts to both immediate and historical context.
Long-term video object segmentation (VOS) remains challenging due to error accumulation under extended occlusions, re-appearance, and scene changes. Although SAM2 provides strong zero-shot performance, its streaming memory can amplify drift over long horizons when recent, unreliable predictions dominate the memory state. We propose SAM2Dual, a training-free, plug-and-play inference-time enhancement that improves long-video robustness without updating model weights. SAM2Dual introduces a Dual Memory design that explicitly separates (i) short-term memory for rapid local adaptation and (ii) long-term memory built via interval-based sampling to preserve global identity cues, combined through a gated fusion strategy. In addition, we present Text-Aware Memory (TAM), which extracts a compact word-level cue from early frames and uses text embeddings to reweight memory contributions based on semantic compatibility, supporting identity preservation when visual evidence becomes weak or ambiguous. Across long-term benchmarks, SAM2Dual consistently improves stability on long videos, raising J&F from 49.33 to 50.65 on MOSEv2 and achieving consistent gains on LVOSv2.
