Search papers, labs, and topics across Lattice.
This paper introduces EventCoT, a novel event-centric video chain-of-thought framework designed to enhance reasoning temporal localization (RTL) by jointly generating answers and their supporting time intervals. By employing event-centric tokenization, EventCoT efficiently identifies relevant events within videos and utilizes embedding matching to ground answers in precise temporal contexts. The framework achieves state-of-the-art performance on the ActivityNet-RTL dataset while significantly reducing the number of visual tokens used compared to prior methods, and demonstrates strong zero-shot capabilities on the ReXTime benchmark.
EventCoT achieves state-of-the-art RTL performance with fewer visual tokens, revolutionizing how we approach temporal reasoning in videos.
Reasoning temporal localization (RTL) requires a model to generate an answer that itself contains the time interval supporting it, so high-level reasoning and precise temporal grounding must be produced jointly in a single response. To tackle this challenging task, we propose the first event-centric video chain-of-thought framework, dubbed EventCoT. EventCoT first performs event-centric tokenization of the input video to convert it into compact event tokens, enabling efficient identification of question-relevant events. It then reasons within the identified events to generate the answer, grounding the time interval via embedding matching that aligns placeholder tokens with visual embeddings. EventCoT achieves state-of-the-art results on ActivityNet-RTL for reasoning temporal localization while using substantially fewer visual tokens than previous work. To verify its general performance, we further evaluate EventCoT on the grounded video question answering benchmark ReXTime, where it attains strong zero-shot results.
