Search papers, labs, and topics across Lattice.
This paper introduces MARE, a novel approach for explainable deepfake detection that leverages vision-language models (VLMs) to generate text-spatially aligned reasoning. MARE incorporates reinforcement learning from human feedback (RLHF) with comprehensive reward functions to incentivize the generation of reasoning content aligned with human preferences, and introduces a forgery disentanglement module to capture forgery traces from facial semantics. Experiments demonstrate that MARE achieves state-of-the-art performance in deepfake detection accuracy and reliability, as well as improved reasoning quality.
By aligning visual and textual reasoning with human preferences, MARE offers a more accurate and explainable approach to deepfake detection, moving beyond simple classification.
Deepfake detection is a widely researched topic that is crucial for combating the spread of malicious content, with existing methods mainly modeling the problem as classification or spatial localization. The rapid advancements in generative models impose new demands on Deepfake detection. In this paper, we propose multimodal alignment and reinforcement for explainable Deepfake detection via vision-language models, termed MARE, which aims to enhance the accuracy and reliability of Vision-Language Models (VLMs) in Deepfake detection and reasoning. Specifically, MARE designs comprehensive reward functions, incorporating reinforcement learning from human feedback (RLHF), to incentivize the generation of text-spatially aligned reasoning content that adheres to human preferences. Besides, MARE introduces a forgery disentanglement module to capture intrinsic forgery traces from high-level facial semantics, thereby improving its authenticity detection capability. We conduct thorough evaluations on the reasoning content generated by MARE. Both quantitative and qualitative experimental results demonstrate that MARE achieves state-of-the-art performance in terms of accuracy and reliability.
