Search papers, labs, and topics across Lattice.
This paper introduces the UVIF framework, which enhances video face forgery detection by utilizing additional annotated images for fine-grained supervision, addressing the challenge of detecting partially forged videos. By employing a unified encoder and a multi-task learning approach, UVIF effectively models both facial videos and images, incorporating a pseudo labeling process and a video-oriented feature alignment strategy. Experimental results show that UVIF significantly outperforms existing methods in detecting partial forgeries without incurring extra computational costs.
Detecting partially forged videos is now feasible with a novel framework that leverages static images for enhanced supervision and accuracy.
Face forgery detection is crucial for preserving the security and integrity of facial data given the rapid developments in face manipulation techniques and deep generative models. Existing methods for video face forgery detection typically assume that all frames in a forged video are manipulated, while detecting partially forged videos that contain only a subset of altered frames remains challenging. To address this issue, we propose a novel framework, UVIF, that utilizes additional annotated images to provide fine-grained supervision for detecting partial forgeries in videos. UVIF employs a unified encoder and a multi-task learning paradigm to jointly model facial videos and images for boosted video face forgery detection. A 2D backbone with temporal fusion modules is employed as the unified encoder. A pseudo labeling process is designed for video frames to bridge their representations with those of static images. A video-oriented feature alignment strategy is further introduced to reduce the distribution gap between videos and images. Extensive experiments on benchmark datasets demonstrate the effectiveness of our framework, which outperforms state-of-theart methods in detecting partially forged videos while introducing no additional computational overhead. Our code is available at https://github.com/haotianll/UVIF.
Tsinghua AI