Search papers, labs, and topics across Lattice.
This paper introduces a Multi-Scale Cross-Modal Transformer (MSCT) to improve audio-visual deepfake detection by enhancing feature extraction and modal alignment. MSCT employs multi-scale self-attention to integrate features of adjacent embeddings within each modality and differential cross-modal attention to fuse audio and visual features. Experiments on the FakeAVCeleb dataset show that MSCT achieves competitive performance, demonstrating improved deepfake detection capabilities.
By better aligning audio and visual features, MSCT significantly boosts deepfake detection, outperforming existing methods on the FakeAVCeleb dataset.
Audio-visual deepfake detection typically employs a complementary multi-modal model to check the forgery traces in the video. These methods primarily extract forgery traces through audio-visual alignment, which results from the inconsistency between audio and video modalities. However, the traditional multi-modal forgery detection method has the problem of insufficient feature extraction and modal alignment deviation. To address this, we propose a multi-scale cross-modal transformer encoder (MSCT) for deepfake detection. Our approach includes a multi-scale self-attention to integrate the features of adjacent embeddings and a differential cross-modal attention to fuse multi-modal features. Our experiments demonstrate competitive performance on the FakeAVCeleb dataset, validating the effectiveness of the proposed structure.
