Search papers, labs, and topics across Lattice.
Tsinghua AIByteDanceCentral South UniversityHUSTNankai UniversitySEUSouth China Normal UniversitySupermicro Computer IncUniversity of CaliforniaUniversity of ZaragozaWHUJun 25, 2026arXiv:2606.27187This paper introduces HarmVideoBench, a comprehensive benchmark designed to evaluate large vision-language models (LVLMs) on their understanding of harmful videos by assessing three hierarchical dimensions: Observable Evidence, Clip-Internal Meaning, and Beyond-Clip Reasoning. The authors critique existing benchmarks for their binary classification approach, which fails to capture the nuanced and layered characteristics of harmful content, and they highlight the lack of explanatory rationales in current evaluation frameworks. Through extensive evaluation of 19 leading models, the study demonstrates that their benchmark-aligned method, BCR, significantly enhances performance in harmful video understanding, achieving a macro average accuracy of 84.4%, up from 61.7%.
HarmVideoBench reveals that existing benchmarks miss critical layers of harmful video understanding, while a new method boosts model accuracy by over 20%.
Large vision-language models (LVLMs) have recently shown immense potential in automated content moderation, sparking growing interest in developing harmful-video benchmarks. However, we identify two primary limitations in existing works: 1) The multi-layered characteristics of harmful videos are overlooked. Existing benchmarks predominantly formulate evaluation as a binary classification task, failing to capture implicit or deep contextual harms. 2) Explanatory rationales are completely absent. Current frameworks measure exclusively whether a model flags a video correctly rather than explaining why, turning evaluation into a black box where models can succeed through superficial shortcuts. To address these problems, we present HarmVideoBench, a multi-layered diagnostic benchmark comprising 1,379 videos paired with 4,137 multiple-choice questions. HarmVideoBench benchmarks three hierarchical dimensions: Observable Evidence, Clip-Internal Meaning, and Beyond-Clip Reasoning, aiming to evaluate models' deep understanding beyond surface cues with carefully balanced and curated samples. We evaluate 19 leading models on HarmVideoBench to assess their multidimensional understanding of harmful videos. Moreover, we introduce BCR, a benchmark-aligned method that predicts reasoning boundaries and dynamically retrieves context only when needed. Experimental results show that BCR substantially improves the base model's performance in harmful video understanding, raising the macro average from 61.7 percent to a state-of-the-art 84.4 percent.
