Search papers, labs, and topics across Lattice.
This study investigates the impact of bilingual fine-tuning on automatic speech recognition (ASR) systems for low-resource languages by evaluating performance across nine diverse language pairs. By incorporating a language identification token during training and inference, the research reveals that high language identification accuracy significantly enhances ASR outcomes, while low accuracy necessitates the inclusion of the token at inference to mitigate performance drops. The findings underscore the importance of language identification in improving ASR systems in multilingual contexts, particularly for underrepresented languages.
Bilingual fine-tuning can dramatically enhance ASR performance in low-resource languages, but only if language identification is accurately managed.
This study explores how bilingual fine-tuning affects automatic speech recognition (ASR) in low-resource languages. We evaluate this method across nine linguistically and geographically diverse language pairs, covering a range of language families and writing systems. To distinguish the two languages, during training, we pre-pend each input text with a language identification token. At inference, the model jointly predicts both the language and transcription from the speech input alone. As texts for which the language is incorrectly determined show low ASR performance, we also conduct a follow-up experiment in which the language identification token is provided both during training and inference. Our results show that bilingual fine-tuning can be beneficial when language identification accuracy is high, and that in cases where language identification performance is low, including the language identification token at inference helps to improve ASR performance.
