Search papers, labs, and topics across Lattice.
This study conducts a controlled audit of two training-free AI-generated image detection scores鈥擜EROBLADE and RIGID鈥攁cross a benchmark of 1,500 images from seven generators, revealing significant vulnerabilities in their performance. Key findings include the influence of implementation details on detection accuracy, the hyperparameter sensitivity of score direction, and the impact of dataset format bias on robustness claims. The results highlight that naive combination methods fail to leverage the complementary strengths of the scores, suggesting a need for more sophisticated approaches to improve detection efficacy.
Implementation details can drastically alter the performance of training-free image detectors, with hyperparameters flipping conclusions about their effectiveness.
Training-free detectors of AI-generated images promise generator-agnostic deployment without classifier training, yet their reported numbers are rarely compared under a single controlled protocol. We audit two representative training-free scores -- an autoencoder-reconstruction score (AEROBLADE-style) and a noise-perturbation feature-similarity score (RIGID-style) -- plus a naive feature-kNN control, on a common 1,500-image GenImage-derived benchmark spanning seven generators and JPEG compression at quality 70 and 50. The audit yields three cautionary findings. (i) Implementation details masquerade as method differences: replacing the LPIPS backbone (AlexNet -> VGG-16) changes overall AUROC by +0.085, and switching between resize-to-512 and native-resolution preprocessing flips per-generator conclusions by up to 0.38 AUROC. (ii) Score direction is not a property of the method but of its hyperparameters: the RIGID-style score is inverted (AUROC < 0.5) on SD1.5 and Wukong at noise level sigma=0.05, recovers to >0.5 for every generator at sigma=0.01, and collapses to 0.15 at sigma=0.3. (iii) Dataset format bias inflates robustness claims: without unified re-encoding, AUROC under JPEG-50 exceeds the clean condition for the AlexNet-backbone reconstruction score; after bias correction the residual anomaly localizes to a single generator (BigGAN). The audited scores have complementary per-generator failure sets, but naive z-score fusion does not beat the best single score, indicating that exploiting complementarity requires direction-aware combination.
