Search papers, labs, and topics across Lattice.
This paper introduces UAV Open-Vocabulary Video Instance Segmentation (UAV-OVVIS), a novel task aimed at enhancing the flexibility and granularity of target detection in UAV videos by allowing open-vocabulary queries. The authors propose AeroTrack, a training-free framework that utilizes periodic open-vocabulary detection and mask propagation to achieve instance-level segmentation with consistent identities across video segments. Experimental results demonstrate that AeroTrack significantly outperforms existing video instance segmentation methods in UAV contexts, showcasing its robustness and generalization capabilities.
UAV-OVVIS enables flexible target detection in UAV videos, outperforming traditional methods by allowing open-vocabulary queries for instance-level segmentation.
Unmanned Aerial Vehicle (UAV) videos are widely used in traffic monitoring, urban management, and emergency rescue. However, existing UAV video perception mainly relies on box-level localization and trajectory association under predefined categories, making it difficult to simultaneously support flexible queries and fine-grained instance-level dynamic understanding in open scenarios. To this end, we introduce a new task, UAV Open-Vocabulary Video Instance Segmentation (UAV-OVVIS), which discovers targets in UAV videos according to open-vocabulary queries and outputs instance-level segmentation trajectories with globally consistent identities. Considering the scarcity of instance-level annotations in UAV scenarios, we propose AeroTrack, a training-free unified framework. AeroTrack centers on periodic open-vocabulary detection, short-segment mask propagation, and cross-segment identity unification, reusing existing visual foundation models to enable UAV-OVVIS. Based on this framework, we instantiate five AeroTrack variants and construct AeroVIS, an evaluation benchmark for UAV-OVVIS containing 9 UAV object categories and 8,279 trajectories. Experiments show that AeroTrack substantially outperforms existing general video instance segmentation methods in UAV scenarios and demonstrates strong open-vocabulary robustness and generalization. To support future research, we release AeroTrack and AeroVIS as a unified framework and benchmark for UAV-OVVIS.
