Search papers, labs, and topics across Lattice.
Tsinghua AIBeihangBeijing Shanwei Zhixing Technology Co.Corresponding authorHKUJun 4, 2026arXiv:2606.05736This paper introduces VTI-CoT, a Visual-Textual Interleaved Chain of Thought framework designed to enhance video reasoning by integrating visual frames with textual reasoning steps. The method addresses the limitations of existing CoT approaches that primarily rely on text, thereby improving the understanding of complex temporal events and causal relationships in videos. Experimental results show that VTI-CoT not only achieves state-of-the-art performance among similarly scaled models but also enhances training efficiency through innovative compression techniques.
Integrating visual frames with textual reasoning steps, VTI-CoT achieves state-of-the-art video reasoning performance while boosting training efficiency.
Video reasoning aims to understand complex temporal events and causal relationships within videos. Recently, Chain-of-Thought (CoT) has been introduced to this field to enhance reasoning accuracy. However, existing CoT-based video reasoning methods primarily rely on text-only information for logical deduction, overlooking critical visual information during the inference process. Inspired by the human cognitive mechanism of reviewing visual segments during inference, we propose VTI-CoT, a Visual-Textual Interleaved CoT framework. VTI-CoT integrates textual reasoning steps with corresponding visual frames. Given the scarcity of visual-textual interleaved CoT in existing datasets, we develop an automated annotation pipeline to construct high-quality multimodal CoT data. Further, reasoning over long-form videos entails increasingly long CoT token sequences, which severely hinders training convergence and efficiency. To address this, we employ Optical Character Recognition (OCR)-based compression techniques to compress CoT supervision signals into a single canvas. Experimental results demonstrate that VTI-CoT achieves state-of-the-art performance among models of the same parameter scale while significantly improving training efficiency.
