Search papers, labs, and topics across Lattice.
This paper introduces TimeProVe, a hybrid framework designed for efficient Long Video Question Answering (LVQA) that balances computational cost with performance by first generating action-grounded hypotheses through lightweight modules before verifying them with a large vision-language model (VLM). The framework's Action-based Candidate Evidence (ACE) module effectively translates temporally localized actions into query-specific candidate answers and evidence windows, significantly improving the identification of relevant information in lengthy videos. Experimental results demonstrate that TimeProVe surpasses the leading baseline on the OpenTSUBench by 7.3%, while also reducing VLM calls by 75% and inference costs by 93%.
TimeProVe achieves a remarkable 7.3% performance boost on long video reasoning tasks while slashing inference costs by 93%.
Long Video Question Answering (LVQA) requires identifying sparse, query-relevant evidence within hours-long untrimmed videos. Existing approaches either process videos densely with large vision-language models (VLMs), incurring prohibitive computational cost, or rely on sparse caption-based reasoning, which often misses temporally localized and motion-centric evidence. We introduce TimeProVe, a cost-efficient hybrid framework for temporally grounded reasoning in long videos. TimeProVe first employs lightweight modules to generate action-grounded answer--evidence hypotheses and subsequently invokes an expensive VLM only for targeted verification. The core of our framework lies in the Action-based Candidate Evidence (ACE) module, which converts temporally localized actions into query-conditioned candidate answers and supporting evidence windows through lightweight LLM reasoning. We further introduce OpenTSUBench (OTB), an open-ended benchmark designed to evaluate temporally grounded reasoning in real-world Activities of Daily Living (ADL) scenarios. Experiments show that TimeProVe outperforms the strongest baseline on OTB by 7.3%, while reducing VLM calls by 75% and inference cost by 93%. Furthermore, without explicit temporal grounding training, TimeProVe achieves competitive performance on Charades-STA, and reaches state-of-the-art results when enhanced with grounding VLMs.
