Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces ArtBoost, a data augmentation strategy that enhances acoustic-to-articulatory inversion (AAI) models by utilizing large-scale speech-mesh datasets to generate pseudo articulatory trajectories. This approach addresses the limitations of costly and sparse electromagnetic articulography (EMA) data by pre-training AAI models on synthetic data before fine-tuning with real EMA data. Experimental results demonstrate significant improvements in performance metrics such as PCC and RMSE, confirming that the synthetic trajectories effectively capture meaningful articulatory dynamics and can be integrated into various AAI architectures.
ArtBoost reveals that leveraging synthetic data from speech-mesh datasets can significantly enhance the performance of AAI models, even with limited real-world EMA data.
Recent acoustic-to-articulatory inversion (AAI) models rely on electromagnetic articulography (EMA) data, which are costly and limited in scale. To address this limitation, we propose \textit{ArtBoost}, a novel data augmentation strategy that leverages large-scale speech--mesh datasets originally developed for speech-driven 3D facial animation to improve AAI under limited EMA supervision. \textit{ArtBoost} extracts pseudo articulatory trajectories from visible facial anchors and uses them for pre-training before fine-tuning on real EMA data. Experiments show consistent improvements in PCC and RMSE. Trajectory analyses confirm that the pseudo articulatory signals reflect physically meaningful visible articulatory dynamics. Additional evaluations across different AAI architectures demonstrate stable performance gains, indicating that \textit{ArtBoost} can be integrated into diverse AAI models. These results suggest that speech--mesh data provide an effective and scalable source of articulatory supervision for AAI. Project page: https://cau-irislab.github.io/Interspeech26-ArtBoost/