Search papers, labs, and topics across Lattice.
This paper introduces ST-GD, a parameter-efficient framework that adapts pre-trained 2D visual-language models like Grounding DINO to spatio-temporal video grounding tasks. To combat data scarcity, ST-GD freezes the base model and injects lightweight adapters (~10M parameters) to instill spatio-temporal awareness, along with a novel temporal decoder for boundary prediction. Experiments on HC-STVG v1/v2 and VidSTG datasets demonstrate that ST-GD achieves competitive performance in data-scarce scenarios while maintaining robust generalization.
Achieve strong spatio-temporal video grounding with only 10M trainable parameters by smartly adapting pre-trained 2D visual-language models.
Spatio-temporal video grounding (STVG) aims to localize queried objects within dynamic video segments. Prevailing fully-trained approaches are notoriously data-hungry. However, gathering large-scale STVG data is exceptionally challenging: dense frame-level bounding boxes and complex temporal language alignments are prohibitively expensive to annotate, especially for specialized video domains. Consequently, conventional models suffer from severe overfitting on these inherently limited datasets, while zero-shot foundational models lack the task-specific temporal awareness needed for precise localization. To resolve this small-data challenge, we introduce ST-GD, a data-efficient framework that adapts pre-trained 2D visual-language models (e.g., Grounding DINO) to video tasks. To avoid destroying pre-trained priors on small datasets, ST-GD keeps the base model frozen and strategically injects lightweight adapters (~10M trainable parameters) to instill spatio-temporal awareness, alongside a novel temporal decoder for boundary prediction. This design naturally counters data scarcity. Consequently, ST-GD excels in data-scarce scenarios, achieving highly competitive performance on the limited-scale HC-STVG v1/v2 benchmarks, while maintaining robust generalization on the VidSTG dataset. This validates ST-GD as a powerful paradigm for complex video understanding under strict small-data constraints.
