Search papers, labs, and topics across Lattice.
This study systematically compares two pre-trained speech encoders, Whisper and FastConformer, for large-scale spoken language identification (LID) across 42 Indic languages, addressing challenges like phonetic overlap and limited labeled data. Evaluating both encoders in frozen and fine-tuned settings with three training objectives, the research finds that the frozen FastConformer encoder achieves over 90% macro accuracy on out-of-domain benchmarks, significantly outperforming Whisper, while hierarchical softmax consistently yields better results than cross-entropy and supervised contrastive loss. Notably, the findings reveal that contrastive objectives can hinder generalization, particularly for low-resource languages, highlighting the complexity of language discrimination within the Central Indo-Aryan family.
FastConformer achieves over 90% accuracy on out-of-domain language identification, outperforming Whisper without task-specific adaptation.
Spoken language identification (LID) for Indian languages is a challenging problem due to the large number of languages, significant phonetic overlap among related varieties, and the scarcity of labeled data for many low-resource languages. In this work, we present a systematic comparative study of two pre-trained speech encoders -- Whisper and FastConformer -- combined with a linear classifier for large-scale Indic LID spanning 42 languages across four linguistic families. We evaluate both encoders in frozen (linear probing) and fine-tuned settings, and compare three training objectives: cross-entropy (CE), supervised contrastive loss with cross entropy (CE + supCon), and hierarchical softmax (HSM). Models are trained on the Vaani dataset and evaluated in a cross-corpus setting on Vaani-Test (held-out), FLEURS, and Kathbath, providing insights into domain generalization. The frozen FastConformer encoder achieves over 90\% macro accuracy on FLEURS and Kathbath without any task-specific adaptation, substantially outperforming Whisper on out-of-domain benchmarks, while fine-tuned Whisper yields stronger in-domain performance. HSM consistently outperforms CE and CE+SupCon for both encoders across all benchmarks, with the largest gains on out-of-domain test sets. CE+SupCon degrades FastConformer's cross-corpus generalization, suggesting that the contrastive objective over-specializes representations to in-domain conditions. Per-family analysis shows that Central Indo-Aryan varieties are the hardest to discriminate, with Hindi--Urdu and the Sadri--Chhattisgarhi--Surgujia cluster being the dominant confusion pairs.
