Search papers, labs, and topics across Lattice.
The paper introduces SciFigDetect, a new benchmark for detecting AI-generated scientific figures, addressing the limitations of current AI-generated image detection methods on structured, text-dense scientific visuals. They created the benchmark using an agent-based data pipeline that retrieves papers, understands text/figures, synthesizes figures, and refines them through review. Benchmarking existing detectors reveals significant failures in zero-shot transfer, generator-specific overfitting, and fragility to image degradation, highlighting a gap in current AIGI detection capabilities.
Current AI-generated image detectors crumble when faced with the structured complexity of scientific figures, revealing a blind spot in our ability to maintain research integrity.
Modern multimodal generators can now produce scientific figures at near-publishable quality, creating a new challenge for visual forensics and research integrity. Unlike conventional AI-generated natural images, scientific figures are structured, text-dense, and tightly aligned with scholarly semantics, making them a distinct and difficult detection target. However, existing AI-generated image detection benchmarks and methods are almost entirely developed for open-domain imagery, leaving this setting largely unexplored. We present the first benchmark for AI-generated scientific figure detection. To construct it, we develop an agent-based data pipeline that retrieves licensed source papers, performs multimodal understanding of paper text and figures, builds structured prompts, synthesizes candidate figures, and filters them through a review-driven refinement loop. The resulting benchmark covers multiple figure categories, multiple generation sources and aligned real--synthetic pairs. We benchmark representative detectors under zero-shot, cross-generator, and degraded-image settings. Results show that current methods fail dramatically in zero-shot transfer, exhibit strong generator-specific overfitting, and remain fragile under common post-processing corruptions. These findings reveal a substantial gap between existing AIGI detection capabilities and the emerging distribution of high-quality scientific figures. We hope this benchmark can serve as a foundation for future research on robust and generalizable scientific-figure forensics. The dataset is available at https://github.com/Joyce-yoyo/SciFigDetect.
