Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces TempoGround, a framework designed to enhance visual grounding in streaming inputs by addressing issues such as identity drift and cross-frame inconsistency. By implementing a curriculum prediction mechanism that models object presence states and utilizing Streaming Grounding Reinforcement (SGR) for optimization, TempoGround achieves significant improvements in both 2D and 3D visual grounding metrics. The results show an average increase of 4.4 in F1_2D@0.5 and 6.2 in F1_3D@0.25 across multiple benchmarks, underscoring its effectiveness in real-time applications.
TempoGround resolves identity drift in streaming visual grounding, achieving unprecedented accuracy in 2D and 3D object localization.
Visual grounding maps language referents to spatial targets and is central to open-vocabulary perception with vision-language models. Existing methods have made substantial progress on single-frame and video-based visual grounding, yet under streaming inputs they still suffer from identity drift, cross-frame inconsistency, and fragile localization under partial occlusion. To address these issues, we present TempoGround, a VLM-native framework that detects cross-frame object correspondence and explicitly models object presence states, thereby enabling accurate and consistent visual grounding under streaming inputs. The key is a curriculum prediction mechanism guided by state-aware cross-frame correspondence: TempoGround resolves 2D instance association, predicts whether each object newly enters, continues in, or leaves the view, decodes the 2D box, and then lifts it to a camera-frame 3D box. As token-level supervision alone cannot capture the geometric objectives of streaming grounding, we further introduce Streaming Grounding Reinforcement (SGR), which optimizes TempoGround with verifiable Grounding, Identity, and Consistency rewards, jointly reinforcing persistent localization and temporally consistent predictions. We carefully design a three-stage training strategy and train TempoGround on large-scale data. We evaluate visual grounding under causally streaming inputs on multiple challenging benchmarks: TempoGround improves F1_2D@0.5 and F1_2D@0.95 by 4.4 and 0.5 on average, and F1_3D@0.25 and AP_3D by 6.2 and 7.5, respectively. These results demonstrate that TempoGround provides a practical foundation for visual grounding under streaming inputs.