Search papers, labs, and topics across Lattice.
This paper introduces Forged Calamity, a benchmark dataset designed for evaluating synthetic disaster detection, comprising 30,000 images—6,000 real and 24,000 synthetic—generated by four different diffusion models. The research reveals that existing forensic methods exhibit significant weaknesses, with fine-tuned detectors suffering up to 50% accuracy loss on unseen disaster types and zero-shot detectors showing limited robustness. These findings underscore the critical need for more effective, domain-agnostic detection approaches to combat the challenges posed by photorealistic synthetic imagery in disaster scenarios.
Current forensic methods fail dramatically against unseen synthetic disasters, revealing a critical vulnerability in our ability to discern reality from AI-generated fabrications.
The rapid advancement of text-to-image diffusion models has enabled the creation of highly photorealistic synthetic images that closely resemble real photographs, making it increasingly difficult to distinguish authentic content from AI-generated fabrications. This poses challenges for cybersecurity, digital forensics, and disaster response, where fake imagery of floods, fires, or earthquakes can spread misinformation or disrupt emergency operations. To address this, we introduce Forged Calamity, a benchmark dataset for synthetic disaster detection containing 30,000 images, including 6,000 real and 24,000 synthetic samples generated by four diffusion models. Comprehensive experiments across fine-tuned and zero-shot settings reveal consistent weaknesses in current forensic approaches. Fine-tuned detectors perform well in-distribution but lose up to 50\% accuracy on unseen generators or disaster types, showing overfitting to model-specific artifacts. Zero-shot generalized detectors also struggle to maintain stable accuracy, with only limited resilience in a few representation-robust models. These findings highlight persistent generalization gaps and the urgent need for domain- and model-agnostic detection methods to ensure visual authenticity in the diffusion era.
