Search papers, labs, and topics across Lattice.
LaP-Forensics introduces a novel multimodal framework for deepfake detection that integrates RGB image semantics with reconstruction-based forensic evidence from a frozen Stable Diffusion model. By employing a structured Where-What-Why model and Group Relative Policy Optimization (GRPO), the framework enhances detection capabilities through a combination of mask overlap and evidence-reference terms. Experimental results demonstrate its effectiveness in cross-generator detection and artifact localization, outperforming existing methods on key benchmarks while highlighting areas for improvement in textual faithfulness and reliability.
LaP-Forensics reveals that leveraging reconstruction-based evidence can significantly enhance deepfake detection accuracy against state-of-the-art generative models.
Recent generative models can produce images with few obvious visual artifacts, weakening detectors and explanations that rely only on surface appearance. We present LaP-Forensics, a multimodal framework that augments RGB semantics with reconstruction-based forensic evidence. A frozen Stable Diffusion DDIM inversion-reconstruction model provides a fixed reconstruction reference, and its residual map measures local compatibility with that reference. Independent projectors encode the RGB image and residual map before a structured Where-What-Why model predicts a textual analysis and an artifact mask.Supervised fine-tuning is followed by Group Relative Policy Optimization (GRPO), whose reward combines mask overlap with output-structure and evidence-reference terms. These text-side terms encourage the model to refer to the consistency map but do not constitute a verifier of free-form textual truth. A separate image-level head fuses RGB and DDIM-residual class features. Experiments show cross-generator detection on UniversalFakeDetect and competitive artifact localization on the official SynthScars benchmark. Controlled cue-construction, inversion-horizon, component, reward-term, and counterfactual analyses support the utility of the residual stream under the evaluated settings, while free-form textual faithfulness and reliability under post-processing remain open limitations.
NUS
Tsinghua AI