Search papers, labs, and topics across Lattice.
This paper introduces TimeLens2, a multimodal large language model designed for generalist video temporal grounding, which effectively identifies and predicts variable-cardinality sets of evidence intervals in videos. The authors address the limitations of existing training strategies by employing a novel approach that treats temporal evidence as interval sets, utilizing advanced techniques such as caption-derived proposals and a temporal Wasserstein reward for optimization. TimeLens2 achieves superior performance across seven benchmarks, outperforming all size-matched baselines and setting new state-of-the-art results, even against models with significantly larger parameters.
TimeLens2 outperforms models with up to 397B parameters by effectively grounding temporal evidence in videos, redefining expectations for multimodal LLM capabilities.
Video multimodal large language models (MLLMs) can describe what happens in a video, but rarely identify when the supporting evidence occurs. We study generalist video temporal grounding, in which one model predicts a variable-cardinality set of evidence intervals across video lengths, domains, query forms, and viewpoints. Existing training strategies are misaligned with this set-valued task: long-video labels often rely on brittle one-pass annotation, while reinforcement-learning rewards either fail to distinguish non-overlapping predictions or require fragile segment matching. TimeLens2 treats temporal evidence as an interval set throughout supervision and optimization. TimeLens2-93K constructs reliable multi-span supervision through caption-derived proposals, independent localization, cross-agent consensus, semantic verification, and boundary refinement. Our temporal Wasserstein reward computes exact one-dimensional \(W_1\) between uniform distributions over merged interval supports, providing dense, matching-free feedback under unequal cardinalities and equivalent fragmentation; temporal IoU complements it with precise-overlap feedback. Across seven benchmarks, TimeLens2-2B outperforms all size-matched baselines on every benchmark, while the 4B and 8B variants achieve state-of-the-art performance, surpassing open-source models with up to 397B parameters. The 2B, 4B, and 8B variants improve over their Qwen3-VL backbones by 14.2, 13.0, and 18.1 mIoU points, respectively.
