Search papers, labs, and topics across Lattice.
This paper introduces a novel pretraining method for Acoustic-to-Articulatory Inversion (AAI) that utilizes three target representations鈥擯honeme Labels, Articulatory Feature Labels, and Critical-articulator Labels鈥攖o enhance performance in low-resource settings. By eliminating the reliance on Self-Supervised Learning (SSL) feature extractors during inference, the approach significantly reduces computational overhead and latency. Experimental results show consistent improvements in AAI performance across various data conditions, particularly benefiting low-resource scenarios while maintaining accuracy.
Eliminating SSL feature extractors in AAI not only boosts performance in low-resource settings but also slashes inference costs dramatically.
Acoustic-to-Articulatory Inversion (AAI) estimates vocal tract articulator movements from speech, benefiting tasks like ASR, speech synthesis, and speaker verification. While deep learning-based methods (CNNs, RNNs, Transformers) have advanced AAI, recent studies show that Self-Supervised Learning (SSL) features further enhance performance, particularly in low-resource settings. However, SSL feature extractors introduce inference latency and computational overhead. To address this, we propose a novel pretraining method leveraging three target representations-Phoneme Labels, Articulatory Feature Labels, and Critical-articulator Labels-eliminating the need for an SSL extractor during inference. We evaluate our approach against both baseline and SSL-based models across various data conditions. Results demonstrate that our method consistently improves AAI performance, particularly in low-resource scenarios, while significantly reducing inference costs without sacrificing accuracy.
