Search papers, labs, and topics across Lattice.
SAM2Matting introduces a novel tracker-to-matting framework that bridges high-level tracking and low-level matting tasks, addressing the limitations of existing video matting methods. By enhancing foundational trackers with a region-proposal bridge and specialized matting heads, the approach achieves high-fidelity video matting while ensuring temporal consistency. Remarkably, SAM2Matting sets a new state-of-the-art in video matting performance, even when trained solely on image data, showcasing robust generalization across various scenarios.
Achieving state-of-the-art video matting performance with a framework trained exclusively on images reveals the untapped potential of existing trackers in fine-grained tasks.
Despite impressive advances in image matting, video matting remains challenging due to the inherent gap between high-level tracking, which requires frame-wise understanding, and low-level matting, which focuses on extremely fine-grained details. Existing methods attempt this with expensive and narrowly-scoped video matting datasets, which may limit out-of-domain generalization and compromise tracking robustness. We rethink the paradigm with SAM2Matting, a tracker-to-matting framework that advances VOS trackers to high-fidelity video matting. Specifically, it decouples the task by enhancing a foundational tracker (e.g., SAM2, SAM3) with a region-proposal bridge and dedicated matting heads, enabling the uncompromised tracker to handle temporal consistency while the matting components resolve fine-grained details. Notably, despite being trained only on images, SAM2Matting establishes new state-of-the-art performance on video matting, supports diverse prompt types, maintains strong temporal consistency, and demonstrates robust generalization across both human-centric and in-the-wild scenarios.
