Search papers, labs, and topics across Lattice.
This paper introduces a two-stage methodology for selecting layers from self-supervised speech models to enhance the generalizability of audio deepfake detection systems. By employing lightweight XGBoost probes to assess the discriminative power of each transformer layer, the authors identify informative depth zones that significantly improve performance when only the selected layers are utilized in a compact neural classifier. The results demonstrate a 28% relative improvement in equal error rate over prior methods, highlighting the importance of targeted layer selection in achieving robust audio deepfake detection across diverse domains.
Informative layers in speech models cluster in depth zones, and selecting them can boost audio deepfake detection performance by 28% over previous methods.
Audio deepfake detection systems often fail to generalize across domains because they rely on features tied to specific attacks or recording conditions. Self-supervised speech models offer rich multi-layer representations, yet existing approaches either use a single layer or fuse all layers indiscriminately, and only reveal layer importance after training. We propose a model-agnostic, two-stage methodology that identifies informative depth zones before any task-specific model is trained. In the first stage, lightweight XGBoost probes evaluate each transformer layer's cross-domain discriminative power, producing a layer ranking. In the second stage, a compact neural classifier fuses only the selected layers through per-layer attention pooling and a shared bottleneck projection, while the backbone remains frozen. Applied across three backbones, the probing reveals two key findings. First, informative layers cluster in depth zones rather than at uniquely optimal positions: within-zone substitutions fall within multi-seed noise, while zone violations degrade performance by up to 5x. Second, the probing produces backbone-specific selections rather than a fixed layer recipe. On XLS-R-300M, four probing-selected layers with 1.34M trainable parameters achieve 4.94 +/- 0.32% equal error rate on In-The-Wild and 5.07% cross-domain average over four shared datasets, a 28% relative improvement over the best prior frozen-backbone result (Xiao and Vu, 2025) using all 25 layers with identical training data.
