Search papers, labs, and topics across Lattice.
The paper introduces VideoHV-Agent, a multi-agent framework for long video understanding that decomposes video question answering into hypothesis generation, clue derivation, evidence verification, and answer integration. A "Thinker" agent rewrites answer candidates into testable hypotheses, a "Judge" derives discriminative clues, a "Verifier" grounds and tests these clues using localized video content, and an "Answer" agent integrates the validated evidence. Experiments on long-video QA benchmarks demonstrate state-of-the-art accuracy, improved interpretability, and reduced computational cost compared to existing methods.
By explicitly formulating and verifying hypotheses *before* retrieving information, VideoHV-Agent side-steps the semantic drift and correlation-driven errors that plague traditional long-video QA systems.
Long video understanding is challenging due to dense visual redundancy, long-range temporal dependencies, and the tendency of chain-of-thought and retrieval-based agents to accumulate semantic drift and correlation-driven errors. We argue that long-video reasoning should begin not with reactive retrieval, but with deliberate task formulation: the model must first articulate what must be true in the video for each candidate answer to hold. This thinking-before-finding principle motivates VideoHV-Agent, a framework that reformulates video question answering as a structured hypothesis-verification process. Based on video summaries, a Thinker rewrites answer candidates into testable hypotheses, a Judge derives a discriminative clue specifying what evidence must be checked, a Verifier grounds and tests the clue using localized, fine-grained video content, and an Answer agent integrates validated evidence to produce the final answer. Experiments on three long-video understanding benchmarks show that VideoHV-Agent achieves state-of-the-art accuracy while providing enhanced interpretability, improved logical soundness, and lower computational cost. We make our code publicly available at: https://github.com/Haorane/VideoHV-Agent.
