Search papers, labs, and topics across Lattice.
This paper introduces a novel pipeline for open-set synthetic face source attribution that addresses the limitations of existing closed-set methods by enabling the identification of both known and unknown face generators. Utilizing frozen I-JEPA embeddings, the approach combines known generator classification with energy-based out-of-distribution (OOD) rejection and clustering techniques to progressively discover new generators from rejected samples. Experimental results demonstrate high accuracy in closed-set attribution and effective clustering of unknown generators, achieving significant metrics such as 71.25% balanced accuracy and 99.23% purity in incremental settings.
Open-set attribution can now effectively identify and cluster unknown face generators, achieving 71.25% balanced accuracy in real-world scenarios where new models continuously emerge.
Recent advances in generative Artificial Intelligence have made synthetic face images increasingly realistic, creating new challenges for multimedia forensics. Source attribution methods should not only identify the generator of an image when the source is known, but also handle samples produced by previously unseen models. However, most existing approaches address synthetic face attribution in a closed-set setting, where all possible generators are available during training. This assumption does not hold in real-world scenarios, where new generators continuously appear and rejected samples should be organized rather than simply discarded. In this work we propose a pipeline for open-set synthetic face source attribution that combines known generator classification, energy-based OOD rejection, and unknown generator discovery. A classifier is trained on known generators using frozen I-JEPA embeddings, while rejected samples are represented by combining projected I-JEPA features with Forensic Self-Descriptors and then clustered to discover groups of unknown generators. We also extend the discovery stage to an incremental scenario, where rejected samples arrive over time. Experiments on the WILD dataset show that the proposed method achieves 96.73% closed-set attribution accuracy. In the open-set setting, energy-based rejection reaches 71.25% balanced accuracy, while rejected samples are clustered into meaningful unknown-generator groups, obtaining an ARI of 0.81, an NMI of 0.90, and an overall clustering purity of 87.74%. In the incremental setting, the discovered generator space is progressively extended while maintaining a final purity of 99.23%. Cross-dataset experiments suggest that the pipeline can operate beyond the original dataset distribution, although post-processing remains challenging.
