Search papers, labs, and topics across Lattice.
This paper investigates deepfake detection using interpretable bio-behavioral features of facial dynamics, moving beyond black-box deep learning approaches. They extracted temporal features from low-dimensional patterns of facial movement and trained traditional machine learning classifiers, achieving above-chance deepfake classification, particularly for videos with emotive expressions. The study also compared model decisions with human perceptual judgments, revealing both convergence and divergence in detection strategies, suggesting complementary approaches to deepfake detection.
Deepfakes betray themselves through subtle irregularities in the timing of facial movements, especially when expressing emotions, offering a new avenue for detection.
Deepfake detection research has largely converged on deep learning approaches that, despite strong benchmark performance, offer limited insight into what distinguishes real from manipulated facial behavior. This study presents an interpretable alternative grounded in bio-behavioral features of facial dynamics and evaluates how computational detection strategies relate to human perceptual judgments. We identify core low-dimensional patterns of facial movement, from which temporal features characterizing spatiotemporal structure were derived. Traditional machine learning classifiers trained on these features achieved modest but significant above-chance deepfake classification, driven by higher-order temporal irregularities that were more pronounced in manipulated than real facial dynamics. Notably, detection was substantially more accurate for videos containing emotive expressions than those without. An emotional valence classification analysis further indicated that emotive signals are systematically degraded in deepfakes, explaining the differential impact of emotive dynamics on detection. Furthermore, we provide an additional and often overlooked dimension of explainability by assessing the relationship between model decisions and human perceptual detection. Model and human judgments converged for emotive but diverged for non-emotive videos, and even where outputs aligned, underlying detection strategies differed. These findings demonstrate that face-swapped deepfakes carry a measurable behavioral fingerprint, most salient during emotional expression. Additionally, model-human comparisons suggest that interpretable computational features and human perception may offer complementary rather than redundant routes to detection.
