Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces selective mask propagation, an innovative multi-object tracking algorithm that intelligently switches from a lightweight base tracker to a more computationally intensive video object segmentation (VOS) model only when uncertainty in identity assignment is detected. By modifying the base tracker's output only when the VOS model confidently contradicts its predictions, the approach effectively balances efficiency and accuracy, preserving identity in challenging frames without the need for extensive retraining. The method achieves state-of-the-art performance on the SportsMOT benchmark, demonstrating a significant improvement in identity preservation metrics, particularly with an HOTA score of 86.8.
Selective mask propagation allows for real-time multi-object tracking that seamlessly integrates lightweight and heavy models, achieving state-of-the-art results without the computational burden of constant VOS use.
Multi-object tracking has a heavy-tailed difficulty distribution: most frames are easy for a lightweight base tracker, while a small fraction are intrinsically hard. Video object segmentation (VOS) models can often preserve identity through the hard frames where the base tracker fails, but they are much more expensive in compute and memory. We propose selective mask propagation, a tracking algorithm that dispatches from a base tracker to a VOS model only on windows where an assignment-uncertainty signal fires. The base tracker's output is modified only when the VOS model makes a confident prediction that contradicts the base tracker's identity assignment; weak or inconclusive predictions preserve the base output. The method is training-free, treats both the base tracker and the VOS model as black boxes, and can benefit from replacing the VOS component with a more capable model. On DanceTrack, selective mask propagation improves three different base trackers. On SportsMOT, where identity preservation is central to sports analytics, SAM3-Deep-EIoU with global track association achieves state-of-the-art performance on the benchmark with 86.8 HOTA.