Search papers, labs, and topics across Lattice.
This paper introduces OA-VAT, a novel visual active tracking pipeline designed to improve instance-level discrimination and handle occlusions in real-world scenarios. OA-VAT uses instance-aware offline prototype initialization with DINOv3 features, online prototype enhancement with a Kalman filter, and an occlusion-aware trajectory planner trained on a new Planning-20k dataset. Experiments show OA-VAT significantly outperforms state-of-the-art methods on UnrealCV and real-world datasets, achieving 0.93 SR and 90.8% CAR, respectively, while running at 35 FPS on an RTX 3090.
DINOv3 representations and diffusion-based planning enable a visual tracker that's both robust to occlusions and discriminative enough to avoid visually similar distractors.
Visual Active Tracking (VAT) aims to control cameras to follow a target in 3D space, which is critical for applications like drone navigation and security surveillance. However, it faces two key bottlenecks in real-world deployment: confusion from visually similar distractors caused by insufficient instance-level discrimination and severe failure under occlusions due to the absence of active planning. To address these, we propose OA-VAT, a unified pipeline with three complementary modules. First, a training-free Instance-Aware Offline Prototype Initialization aggregates multi-view augmented features via DINOv3 to construct discriminative instance prototypes, mitigating distractor confusion. Second, an Online Prototype Enhancement Tracker enhances prototypes online and integrates a confidence-aware Kalman filter for stable tracking under appearance and motion changes. Third, an Occlusion-Aware Trajectory Planner, trained on our new Planning-20k dataset, uses conditional diffusion to generate obstacle-avoiding paths for occlusion recovery. Experiments demonstrate OA-VAT achieves 0.93 average SR on UnrealCV (+2.2% vs. SOTA TrackVLA), 90.8% average CAR on real-world datasets (+12.1% vs. SOTA GC-VAT), and 81.6% TSR on a DJI Tello drone. Running at 35 FPS on an RTX 3090, it delivers robust, real-time performance for practical deployment.
