Search papers, labs, and topics across Lattice.
This paper introduces TimePLE, a novel approach to video temporal grounding (VTG) that shifts the focus from endpoint prediction to directly predicting valid temporal intervals. By formulating the problem as a joint distribution over intervals, TimePLE improves the representation of interval validity and similarity, addressing limitations of existing methods that treat intervals as derived objects. Experimental results demonstrate that TimePLE achieves an average mIoU of 58.9 across four benchmarks, outperforming traditional endpoint prediction methods, particularly for short and medium-duration events.
TimePLE redefines video temporal grounding by predicting valid intervals directly, leading to significant performance gains over traditional endpoint-based methods.
Video temporal grounding (VTG) aims to localize the continuous video interval described by a natural-language query. However, current VLM-based methods typically produce this interval indirectly through two endpoint outputs, represented either as discrete timestamp tokens or continuous boundary coordinates. These formulations differ in how endpoints are encoded, but not in what is predicted: the event interval remains a derived object, while interval validity, duration, and interval-level similarity are handled only implicitly. We propose TimePLE, which reformulates VTG from endpoint prediction to interval-native grounding by predicting a single joint distribution over valid temporal intervals. TimePLE maps each interval to a point in a canonical position-duration square, where every support point corresponds to a valid span and neighboring points represent geometrically similar intervals. Given a video and query, the VLM generates a single latent <|TIMESPAN|> token whose hidden state is decoded into a joint interval distribution, refined through duration-aware coordinate correction, and converted into continuous boundaries. The same interval representation is used to encode input temporal anchors, aligning video-side temporal evidence with output-side span prediction. To reliably align the latent span representation with complete event intervals, we curate 90K-scale grounded samples and human-verify 3K-scale benchmark annotations. Experiments across four VTG benchmarks show that TimePLE consistently outperforms endpoint prediction baselines, achieving an average mIoU of 58.9, with clear gains on short-duration and medium-duration events.
