Search papers, labs, and topics across Lattice.
This paper introduces TD-VAD, a novel approach to video anomaly detection (VAD) that utilizes text descriptions of abnormal events instead of relying on scarce annotated video data. By employing a text-driven learning framework, the method captures temporal dependencies through an event evolution causal attention module, allowing for effective training without target-domain anomaly data. Experimental results on XD-Violence and UCF-Crime datasets show that TD-VAD significantly outperforms existing one-class and unsupervised VAD methods, highlighting its potential in addressing the data scarcity issue in VAD tasks.
Text descriptions can replace visual data in video anomaly detection, leading to a dramatic boost in performance without the need for annotated video datasets.
Visual data is typically a prerequisite for training existing video anomaly detection (VAD) methods. However, obtaining sufficient annotated anomaly data for training is challenging and not scalable due to the rarity of anomaly data and the wide variety of abnormal events. In this work, we advocate that the effectiveness of treating texts as video sequences for the VAD model and propose a novel Text-Driven Video Anomaly Detection (TD-VAD) approach to break visual dependence. In contrast to the anomaly video data, text descriptions of abnormal events are easy to collect, and their class labels can be directly derived. Specifically, our method utilizes video-like text descriptions with temporal characteristics generated by LLM to train a VAD model, without any reliance on target-domain anomaly data. To capture the long- and short-range temporal logic of events, we design the event evolution causal attention module to model contextual dependencies across time. During inference, considering the domain gap between the texts and video sequences, we use the frozen CLIP encoder to extract embeddings of video frames to align the text modality while retaining crucial visual information. Comprehensive experiments on two large-scale VAD datasets, XD-Violence and UCF-Crime, demonstrate that our method outperforms prior one-class and unsupervised VAD methods by a large margin.
