Search papers, labs, and topics across Lattice.
This paper introduces a unified phoneme-based TTS-to-ASR augmentation pipeline leveraging a multilingual TTS model trained from scratch, which integrates various components such as grapheme-to-phoneme conversion and candidate-text selection. The authors implement phoneme-frequency-guided selection (PFGS) to optimize candidate sentence selection based on phoneme frequencies derived from real ASR training labels. Experimental results demonstrate that random augmentation surpasses matched real-only continuation in 11 out of 13 test sets, with PFGS achieving significant improvements in word error rates, underscoring the importance of synthesis scale and reference quality in TTS-based ASR augmentation.
PFGS can reduce word error rates by up to 19.3% compared to random selection, highlighting the critical role of phoneme frequency in TTS augmentation for ASR.
Synthetic speech provides scalable supervision for automatic speech recognition (ASR), but its benefit depends on the selected texts, reference speech, and amount of synthesized data. We present a unified phoneme-based TTS-to-ASR augmentation pipeline built around a multilingual TTS model trained from scratch using the F5-TTS architecture with language-ID conditioning. The pipeline combines language-specific grapheme-to-phoneme conversion, reference-speech filtering, candidate-text selection, synthesis, and matched ASR continuation. We further propose phoneme-frequency-guided selection (PFGS), which ranks candidate sentences using phoneme frequencies estimated from real ASR training labels. Experiments with separate monolingual ASR systems for Arabic, French, Italian, and Portuguese span 13 test sets. Across the synthesis-scale sweep, random augmentation improves over matched real-only continuation on 11 test sets. Under a nominal 60% synthesis budget, PFGS improves over real-only training on 12 test sets and over random selection on 9. Its largest relative word error rate (WER) reduction against random selection is 19.3%. With target texts and synthesis counts fixed, reference-speech filtering reduces absolute WER by 0.29 and 0.59 points on Italian and French Common Voice, respectively. These results identify synthesis scale, candidate-text content, and reference quality as important control variables in TTS-based ASR augmentation.
