Search papers, labs, and topics across Lattice.
The paper introduces Chain-of-Glimpse, a framework for video understanding that grounds reasoning steps to specific visual regions, addressing limitations of object-agnostic methods. It uses a reinforcement learning-optimized search-guided controller to iteratively identify and ground task-relevant objects, forming reliable reasoning trajectories. Experiments on NExTQA, Video-Holmes, CG-Bench Reasoning, and VRBench show Chain-of-Glimpse achieves consistent performance gains and improved generalization across diverse video reasoning tasks.
By explicitly grounding reasoning steps to visual objects, Chain-of-Glimpse enables more accurate and interpretable video understanding, outperforming object-agnostic methods on multiple benchmarks.
Video understanding requires identifying and reasoning over semantically discriminative visual objects across frames, yet existing object-agnostic solutions struggle to effectively handle substantial object variations over time. To address this, we introduce Chain-of-Glimpse, a search-guided progressive object-grounded reasoning framework that explicitly anchors each reasoning step to specific visual evidence regions, enabling compositional and multi-step decision-making. Formally, Chain-of-Glimpse formulates video reasoning as a step-by-step process that incrementally builds spatially grounded traces around task-relevant visual objects, thereby mitigating over-reliance on saliency-driven cues. Specifically, Chain-of-Glimpse features a search-guided controller, optimized via reinforcement learning with a format reward that significantly incentivizes grounding capability, to iteratively ground visual evidence regions and form reliable reasoning trajectories, yielding accurate and interpretable multi-step decisions. Extensive evaluations on both in domain NExTQA and out-of-domain Video-Holmes, CG-Bench Reasoning, and VRBench benchmarks demonstrate consistent performance gains, robustness and generalization of Chain-of-Glimpse across diverse video reasoning tasks.
