Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a multimodal framework for detecting deepfake news by integrating audio and visual cues through advanced feature extraction techniques. The framework utilizes LipNet and DeepSpeech2 for encoding lip movements and speech, while facial features are captured using BlazeFace and ResNet18, culminating in a comprehensive video representation. Experimental results on the FakeAVCeleb dataset demonstrate a significant accuracy of 94%, surpassing existing state-of-the-art methods and highlighting the framework's effectiveness in combating deepfake threats.
Achieving 94% accuracy in deepfake detection by seamlessly integrating audio and visual analysis could redefine standards for authenticity in digital media.
Deepfake news refers to AI-generated (or AI ma-nipulated) multimedia content intentionally generated to deceive audiences by manipulating the facial expressions, or speech while maintaining the realistic appearance. The rapid progress of generative AI has made the synthesis of highly realistic fake videos and cloned voices widely accessible, posing a serious threat to the authenticity of digital news media. This paper presents a multi-modal framework that discerns the authenticity of video content by jointly exploiting audio and visual cues, thereby addressing the challenge of detecting the deepfake videos. We proposed a framework that involves features extraction from lip movements, audio content and video frames. Lip movements and speech content are encoded using the LipNet and DeepSpeech2 models, while facial features are extracted by leveraging the use of BlazeFace and represented with ResNet18. The extracted feature vectors are concatenated into a holistic video representation and classified with an ensemble of machine learning and deep learning models, including Random Forest (RF), Multi-layer Perceptron (MLP) and Long Short-Term Memory (LSTM) networks. Exten-sive experiments performed on the FakeAVCeleb dataset shows that the proposed approach attains an accuracy of 94% using augmented audio features, outperforming a state-of-the-art multi-modal ensemble baseline. The results confirm the robustness and practical potential of the proposed framework for deepfake news detection.