Search papers, labs, and topics across Lattice.
This study evaluates acoustic-to-articulatory inversion (AAI) performance across Finnish and Russian, addressing the challenges posed by speaker variability and language differences. By establishing baseline benchmarks on the FROST-EMA corpus, the research systematically assesses the impact of articulatory targets, acoustic front-ends, and inversion back-ends on inversion accuracy. Key findings reveal that while cross-gender mismatches result in moderate declines in correlation, cross-language mismatches lead to significantly larger performance drops, highlighting the need for robust models in multilingual contexts.
Cross-language mismatches in acoustic-to-articulatory inversion can degrade performance by up to 20%, underscoring the limitations of current AAI models in multilingual applications.
Acoustic-to-articulatory inversion (AAI) remains challenging under domain shifts where changes in speaker attributes and cross-language conditions often degrade performance. We conduct a systematic evaluation under such shifts and establish baseline benchmarks on FROST-EMA, a Finnish-Russian bilingual EMA corpus. FROST-EMA addresses the English bias and limited speaker diversity of existing resources. We benchmark (i) articulatory targets (raw EMA coordinates vs tract variables), (ii) acoustic front-ends (MFCC vs SSL features), and (iii) inversion back-ends (BiLSTM vs a lightweight attention-based sequence model). We further define evaluation protocols for cross-gender transfer (within language) and cross-language transfer (within gender). The results indicate that cross-gender mismatch introduces moderate Pearson correlation declines (approximately 0.05 to 0.10) relative to the in-domain baseline, whereas cross-language mismatch causes larger drops (approximately 0.10 to 0.20).
