Search papers, labs, and topics across Lattice.
This paper introduces LongVQUBench, a novel benchmark designed to evaluate long-term video quality understanding in large vision-language models (LVLMs), addressing the shortcomings of existing benchmarks that focus on short clips. The benchmark includes over 1200 diverse videos and 1500 questions, structured across three levels of evaluation complexity: local event quality understanding, cross-event quality reasoning, and global quality understanding. Experiments reveal that LVLMs experience significant performance degradation as video length and reasoning complexity increase, underscoring their limitations in long-range temporal integration and perceptual reasoning.
LVLMs falter dramatically in video quality understanding as the duration and complexity of content increase, revealing critical gaps in their perceptual reasoning capabilities.
The evaluation of long-term video quality understanding remains an open challenge for large vision-language models (LVLMs). Existing video quality benchmarks predominantly focus on short clips and isolated distortions, overlooking the temporal continuity, cumulative degradation, and reasoning complexity inherent in long-duration content. To address these limitations, we present LongVQUBench, a comprehensive benchmark for long-term video quality understanding. LongVQUBench contains over 1200 diverse videos spanning movies, documentaries, surveillance footage, egocentric recordings, and animated content, accompanied by 1500 multiple-choice and open-ended questions for validation and testing. To assess perceptual reasoning across different temporal scopes, we introduce three progressively complex evaluation levels: (i) local event quality understanding (LQU) for analyzing localized distortions; (ii) cross-event quality reasoning (CQR) for integrating multiple degraded events; and (iii) global quality understanding (GQU) for holistic perceptual evaluation over extended durations. Furthermore, a needle distortion question-answering (NDQA) paradigm is embedded across all three levels, where spatial or temporal artifacts are sparsely inserted to probe fine-grained detection and reasoning capabilities. Extensive experiments on 14 state-of-the-art LVLMs reveal significant performance degradation with increasing video length and reasoning depth, highlighting their limited capacity for long-range temporal integration and perceptual attribution. We envision LongVQUBench as a foundational step toward the systematic, hierarchical, and explainable evaluation of LVLMs' long-term video quality understanding.
