Search papers, labs, and topics across Lattice.
The paper introduces the Singing Head DeepFake (SHDF) dataset to address the performance degradation of audio-visual deepfake detection methods when applied to singing, where the typical cross-modal consistency is weakened. They propose a Text-guided Audio-Visual Forgery Detection (T-AVFD) framework that uses a facial authenticity pattern learner aligned with textual descriptions and a multi-modal differential weight learning module. Experiments demonstrate that T-AVFD outperforms existing methods on both talking head deepfake datasets and the new SHDF dataset, showing robustness to perturbations.
Singing deepfakes are a surprisingly effective attack vector, and this paper introduces a new benchmark and detection method to address the resulting security gap.
With rapid advances in audio-visual generative models, reliable forgery detection becomes increasingly critical. Existing methods for audio-visual deepfake detection typically rely on cross-modal inconsistencies. In singing, rhythmic vocalization weakens this coupling and introduces a nontrivial domain shift, substantially degrading detection performance. We construct the Singing Head DeepFake (SHDF) dataset using rhythm-aware generative models to fill the gap in singing benchmarks. To cope with cross-scenario domain shifts, we propose a Text-guided Audio-Visual Forgery Detection (T-AVFD) framework that generalizes across both talking and singing scenarios. T-AVFD comprises a facial authenticity pattern learner and a multi-modal differential weight learning module. The pattern learner aligns facial features with multi-granularity textual descriptions to learn generalizable authenticity patterns. The weight learning module preserves intrinsic audio-visual consistency and adaptively integrates it with authenticity patterns via differential weighting. Extensive experiments on multiple talking head deepfake datasets and SHDF show consistent improvements over existing baselines and strong robustness under diverse perturbations.
