Search papers, labs, and topics across Lattice.
The paper introduces MoLGE, a novel framework for multilingual automatic speech recognition (ASR) that utilizes a Mixture of Language Group Experts to efficiently manage model capacity across hundreds of languages. By clustering similar languages and integrating a hierarchical Low-Rank Adaptation strategy, MoLGE enhances performance while minimizing the increase in trainable parameters. Experimental results on a benchmark of 495 languages reveal that MoLGE significantly outperforms traditional dense multilingual models, highlighting the importance of structured language specialization in scaling ASR systems.
Clustering languages into groups allows MoLGE to outperform traditional multilingual ASR models while keeping the parameter count low.
Massively multilingual automatic speech recognition (ASR) models covering hundreds of languages must maintain robust performance across diverse linguistic and acoustic conditions. However, these models often encounter the curse of multilinguality, where model capacity is diluted across languages. To address this challenge, we propose Mixture of Language Group Experts (MoLGE), built upon speech self-supervised models (S3Ms). MoLGE assigns dedicated expert modules to clusters of similar languages, reducing the number of required submodules compared to conventional language-specific Mixture-of-Experts (MoE) schemes. It further integrates a hierarchical Low-Rank Adaptation (LoRA) strategy into the disentangled acoustic and linguistic components of the S3M architecture, enabling efficient modeling of language-specific characteristics while maintaining parameter efficiency. Further, we investigate the impact of language grouping strategies based on both linguistic and data-driven criteria on overall performance, providing an interpretable perspective on how language structure influences scalability in multilingual speech systems. In experiments, we evaluate MoLGE on a multilingual benchmark encompassing 495 languages. Results demonstrate that MoLGE consistently outperforms dense multilingual baselines with a minimal increase in trainable parameters. Notably, these language grouping strategies yield substantial improvements for both phonetic and orthographic aspects of ASR modeling. Our findings suggest that structured language specialization provides an effective pathway for massively scaling language coverage of multilingual ASR.
