Search papers, labs, and topics across Lattice.
This paper introduces AnyGroundBench, a specialized-domain benchmark aimed at enhancing the evaluation of Vision-Language Models (VLMs) in Spatio-Temporal Video Grounding (STVG). By shifting the focus from traditional zero-shot assessments to rigorous domain adaptation across five specialized fields, the benchmark addresses the limitations of current models in handling rare visual concepts and complex dynamics. The evaluation of 15 state-of-the-art VLMs reveals significant shortcomings in zero-shot generalization and In-Context Learning (ICL) capabilities, highlighting urgent areas for improvement in spatio-temporal reasoning.
Current VLMs struggle with specialized domains, failing to adapt effectively in both zero-shot and ICL scenarios, revealing critical gaps in their spatio-temporal reasoning abilities.
Vision-Language Models (VLMs) have demonstrated immense promise in Spatio-Temporal Video Grounding (STVG). However, current evaluation protocols are largely confined to zero-shot assessments on general, daily-life benchmarks. This creates a critical disconnect from real-world applications in specialized fields, where models inevitably encounter rare visual concepts and complex spatio-temporal dynamics. Since exhaustive pre-training across infinite data distributions is infeasible, the ability to adapt to novel domains is essential. To bridge this gap, we introduce AnyGroundBench, a domain-adaptation benchmark designed to shift the STVG evaluation paradigm from static zero-shot testing to rigorous domain adaptation. Targeting five specialized domains (animal, industry, sports, surgery, and public security), AnyGroundBench pairs newly captured videos such as expert-annotated mouse behaviors with established datasets, unifying them through dense, high-fidelity spatio-temporal annotations. Crucially, the benchmark provides dedicated training subsets to systematically measure domain adaptability. We extensively evaluate 15 state-of-the-art VLMs, assessing their zero-shot generalization and In-Context Learning (ICL) capabilities under practical computational constraints. Ultimately, our findings reveal that current models fail in both zero-shot and ICL-based adaptation when confronted with specialized domains, exposing critical flaws in spatio-temporal reasoning that future research must address.
NVIDIA