Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Linguistically Augmented Audio Speech Data (LinguAS), a dataset designed to enhance the detection of deepfaked audio by incorporating Expert-Defined Linguistic Features (EDLFs) alongside traditional frame-level audio features. By annotating over 800 audio samples with EDLFs and additional metadata such as speaker gender and spoofing source, the authors provide a richer context for training detection models. The results show that models utilizing this linguistically augmented data significantly outperform existing deep learning baselines and self-supervised learning models in detecting spoofed audio attacks.
Models trained on LinguAS outperformed state-of-the-art deep learning baselines by leveraging linguistic features, revealing a critical gap in current audio deepfake detection approaches.
Maliciously-created fake speech, including deepfaked and spoofed audio, is proliferating at an alarming rate, and detection models are racing to stay ahead of the curve. Yet, most detection models are trained to make inference on frame-level audio features alone without leveraging valuable linguistic cues at larger timescales. To address this gap, we present Linguistically Augmented Audio Speech Data (LinguAS), a dataset of genuine and deepfaked audio samples annotated with five strategically-chosen, Expert-Defined Linguistic Features (EDLFs) that occur frequently in spoken English and are characteristic of natural human speech. LinguAS contains over 800 audio samples, each of which are annotated with EDLFs. The dataset has a balanced number of four spoofed audio attack types and a proportionate number of genuine speech samples. We also include metadata on speaker gender and the generator/source for each spoofed audio sample, offering more granularity for model training. We found that models trained on data augmented with EDLFs had improved model performance significantly beyond the ASVspoof 2021 deep learning baselines and SSL models like HuBert and XLSR. LinguAS's augmented linguistic, gender, and generator metadata provide audio deepfake researchers with a dataset that emphasizes real human language traits to improve model inference of faked speech. Data and code are publicly available.