Search papers, labs, and topics across Lattice.
This paper introduces YesTrack, a two-stage method for referring multi-object tracking (RMOT) that reformulates the task as a discriminative Yes/No verification using multimodal large language models (MLLMs), rather than relying on external caption generation. By integrating lightweight temporal consistency constraints, namely Temporal Confidence Prior (TCP) and Temporal Reference Propagation (TRP), YesTrack enhances both the reliability and efficiency of tracking. Experimental results on the Refer-KITTI and Refer-KITTI-V2 datasets demonstrate that YesTrack outperforms existing state-of-the-art methods while maintaining high efficiency, even with the smallest variant of Qwen3-VL.
YesTrack leverages MLLMs for direct Yes/No verification in multi-object tracking, achieving superior performance without the latency of traditional captioning methods.
Referring multi-object tracking (RMOT) aims to track every instance in a video that matches a given language expression. Despite the recent integration of multimodal large language models (MLLMs) to enhance generalization, existing methods predominantly relegate them to the role of caption generators, necessitating external modules for final decision-making. This paradigm not only introduces extra latency but also severely underutilizes the inherent vision-language alignment capabilities of MLLMs. To address these limitations, we propose YesTrack, a novel two-stage RMOT method that reformulates referring as a discriminative task, directly leveraging MLLMs for Yes/No verification without explicit text generation. To further enhance the reliability and efficiency of this MLLM-based verification, we introduce two lightweight temporal consistency constraints: Temporal Confidence Prior (TCP) and Temporal Reference Propagation (TRP). We further validate the generality of this discriminative paradigm by proposing YesTrack-MOT, a straightforward yet highly effective instantiation for generic multi-object tracking (MOT). Experiments on Refer-KITTI and Refer-KITTI-V2 show that YesTrack significantly outperforms existing state-of-the-art methods while maintaining high efficiency, even when implemented with the smallest variant of Qwen3-VL. Code is released at https://github.com/ggbondrighthere24/YesTrack.
