Search papers, labs, and topics across Lattice.
This paper introduces JECA^2, a white-box adversarial attack against forensic vision-language models (VLMs) that aims to not only flip the model's judgment about image authenticity but also to generate explanations consistent with that flipped judgment. JECA^2 uses Grad-CAM guided perturbations to shift visual attribution away from tampered regions and optimizes prompt embeddings to generate authenticity-affirming explanations. Experiments demonstrate that JECA^2 achieves higher attack success and judgment-explanation consistency compared to baselines, revealing a vulnerability in current forensic VLMs.
Forensic vision-language models can be fooled into not only misclassifying tampered images, but also providing plausible, yet incorrect, explanations, highlighting a critical consistency failure.
Forensic vision-language models (VLMs) have recently been developed to detect image tampering and provide natural-language explanations. However, their robustness against adversarial manipulation remains underexplored. Existing adversarial attacks typically aim to flip the model's binary judgment, while the accompanying explanation may still reveal forensic cues and contradict the attacked judgment. In this paper, we study judgment-explanation consistent adversarial attacks against forensic VLMs and propose JECA^2, a controlled white-box red-team diagnostic that jointly redirects visual attribution and aligns textual explanations with the target judgment. On the visual side, JECA^2 uses Grad-CAM-guided perturbations to divert attribution from tampered regions toward benign regions. On the textual side, it optimizes prompt embeddings toward authenticity-affirming semantics under a token-proximity constraint. Experiments on forensic VLM benchmarks show that JECA^2 achieves higher attack success and automated judgment-explanation consistency than implemented baselines under white-box threat settings, while transfer to closed-source VLMs remains measurable but limited. Our results highlight a consistency failure mode in explanation-based forensic VLMs and motivate future robustness evaluation beyond binary detection accuracy.
