Search papers, labs, and topics across Lattice.
This paper introduces the task of Listening Deepfake Detection (LDD), addressing the overlooked scenario where deepfakes alternate between speaking and listening states to enhance realism. To facilitate research in this area, the authors present ListenForge, a novel dataset constructed using five Listening Head Generation (LHG) methods. They further propose MANet, a Motion-aware and Audio-guided Network, which outperforms existing Speaking Deepfake Detection (SDD) models by capturing subtle motion inconsistencies in listener videos and leveraging speaker's audio semantics.
Current deepfake detectors fall flat when faced with forged listening reactions, but a new motion- and audio-aware network can spot the subtle tells.
Existing deepfake detection research has primarily focused on scenarios where the manipulated subject is actively speaking, i.e., generating fabricated content by altering the speaker's appearance or voice. However, in realistic interaction settings, attackers often alternate between falsifying speaking and listening states to mislead their targets, thereby enhancing the realism and persuasiveness of the scenario. Although the detection of'listening deepfakes'remains largely unexplored and is hindered by a scarcity of both datasets and methodologies, the relatively limited quality of synthesized listening reactions presents an excellent breakthrough opportunity for current deepfake detection efforts. In this paper, we present the task of Listening Deepfake Detection (LDD). We introduce ListenForge, the first dataset specifically designed for this task, constructed using five Listening Head Generation (LHG) methods. To address the distinctive characteristics of listening forgeries, we propose MANet, a Motion-aware and Audio-guided Network that captures subtle motion inconsistencies in listener videos while leveraging speaker's audio semantics to guide cross-modal fusion. Extensive experiments demonstrate that existing Speaking Deepfake Detection (SDD) models perform poorly in listening scenarios. In contrast, MANet achieves significantly superior performance on ListenForge. Our work highlights the necessity of rethinking deepfake detection beyond the traditional speaking-centric paradigm and opens new directions for multimodal forgery analysis in interactive communication settings. The dataset and code are available at https://anonymous.4open.science/r/LDD-B4CB.
