Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To overcome the tendency of single vision foundation models to overfit to specific forgery artifacts, UCF-Net cascades hierarchical representations from CLIP's language-aligned features and DINO's self-supervised visual-structure priors. The architecture leverages layer-wise expert aggregation to extract multi-depth transformer cues and arbitrates between the two encoders using entropy-derived uncertainty weighting. Across a consolidated 4-million-image benchmark, the approach achieves state-of-the-art mean AUC in both in-domain and cross-domain settings while demonstrating rapid few-shot adaptation to novel generators.
Single vision foundation models inevitably overfit to deepfake artifacts, but dynamically arbitrating between CLIP's semantics and DINO's structural features via prediction entropy delivers state-of-the-art generalization across unseen generators.
The growing realism and accessibility of manipulated and generated faces threaten the trustworthiness of digital media. To detect such forgeries, deepfake detectors based on vision foundation models have shown promising performance, but they typically rely on a single pretrained representation and are prone to overfitting to particular training distributions. To improve generalization to unseen forgeries, we propose UCF-Net, an uncertainty-aware cascaded fusion network that harnesses CLIP's language-aligned semantic priors and DINO's self-supervised visual-structure priors. UCF-Net extracts hierarchical features across Transformer depths, uses layer-wise expert aggregation to adaptively combine each encoder's multi-level cues, and performs weighted fusion of the resulting representations based on entropy-derived uncertainty. We further consolidate public deepfake datasets into a unified benchmark of approximately 4M images and construct a separate cross-generator evaluation set with over 8K face images from eight recent generators. On the unified benchmark, UCF-Net achieves the best mean AUC among the evaluated methods in both in-domain and cross-domain evaluations. On the cross-generator set, it adapts effectively with limited target-domain data, although zero-shot transfer remains challenging.
Tsinghua AI