Search papers, labs, and topics across Lattice.
This paper introduces CUPID, a novel deepfake detection system specifically designed for identifying manipulated videos of high-profile individuals without requiring any prior exposure to deepfake content or specific persons in the training set. By leveraging UV texture maps derived from 3D face reconstructions and the Masked Autoencoder (MAE), CUPID achieves superior robustness against post-processing techniques while maintaining interpretability through the extraction of residual maps that highlight facial discrepancies. Experimental results demonstrate that CUPID outperforms existing state-of-the-art methods across multiple datasets, achieving faster inference times and enhanced resilience to video quality degradation.
CUPID outperforms existing deepfake detection methods by leveraging 3D facial representations for unmatched robustness and interpretability, even without prior exposure to specific individuals.
Deepfakes targeting a high-profile individual, known as Person-of-Interest (POI), are a threat to modern democracies and societies. Current POI deepfake detection methods still struggle to combine robustness to post-processing, efficiency and interpretability, focal aspects of modern deepfake detectors. In this paper we propose CUPID, a POI video deepfake detector that combines UV texture maps, a facial appearance representation derived from 3D face reconstructions, with the representation learning capabilities of the Masked Autoencoder (MAE). Our method does not require any deepfake videos in its training phase. Moreover, it does not even require to include a specific POI in the training set: the combination of UV texture maps extracted from real video frames and the MAE context-guided reconstruction yields a latent space that captures rich and discriminative facial features also for identities unseen during training. In the testing phase, the embeddings extracted from a query video depicting the POI can be matched against pristine reference videos to assess the video authenticity. Furthermore, operating in the UV space naturally provides an additional layer of interpretability. Specifically, we can extract decoded residual maps that highlight which facial regions of a test video deviate most from the identity representation of the corresponding POI. Experiments on four deepfake datasets show that CUPID outperforms current state of the art on most datasets and achieves the best overall robustness against strong downscaling and compression, providing also substantially faster inference. Our experimental code will be released at https://github.com/polimi-ispl/CUPID.
