Search papers, labs, and topics across Lattice.
This paper introduces DailyBench, a comprehensive benchmark designed to evaluate AI-generated image detectors against both modern generative models and object-level manipulations. By incorporating two distinct subsets, FakeBench and ManipulationBench, DailyBench addresses the inadequacies of existing benchmarks that fail to reflect the complexities of real-world image generation and editing. Experimental results reveal significant performance drops in current detection methods, underscoring the urgent need for improved robustness in AI-generated image detection systems.
Existing AI-generated image detectors falter dramatically, with accuracy plummeting from 91-96% to as low as 54-66% when faced with realistic manipulations.
Recent advances in generative models have shifted AI-generated image detection from identifying easily distinguishable, fully synthetic images to identifying highly realistic content generated by both modern generation and manipulation pipelines. However, existing detection benchmarks are often built with outdated generative models and primarily emphasize full-image synthesis, creating a growing mismatch between benchmark data and the images encountered in real-world generation and editing scenarios. To bridge this gap, we introduce DailyBench, a high-quality unified benchmark for evaluating whether AI-generated image detectors can generalize across both modern full-image synthesis and object-level manipulation. DailyBench contains two complementary subsets: FakeBench, which includes high-quality images synthesized by recent open-source and commercial generative models, and ManipulationBench, which introduces challenging object-level edits applied to real images using advanced image-conditional models. This design makes DailyBench a realistic testbed for studying both generator-level generalization and manipulation-aware detection under subtle local edits. Experiments on DailyBench reveal substantial robustness gaps in current detectors: methods reporting 91-96% balanced accuracy on GenImage drop to 60-76% on FakeBench and 54-66% on ManipulationBench. These results show that existing detectors remain poorly generalized to realistic synthesis and manipulation, highlighting DailyBench as a rigorous testbed for developing robust and manipulation-aware AI-generated image detection methods. The project is available at https://dailybench.github.io/