Search papers, labs, and topics across Lattice.
The authors developed an end-to-end system for long-form Bengali ASR and speaker diarization, addressing challenges like low resources and code-mixing. They fine-tuned a Whisper medium model for ASR, incorporating Demucs source separation and silence-aware chunking, and fine-tuned a pyannote.audio segmentation model for diarization, using wespeaker embeddings and agglomerative clustering. The system achieved a best private WER of 0.37738 and DER of 0.27671, demonstrating the effectiveness of domain-specific fine-tuning, source separation, and silence-aware chunking.
Domain-specific fine-tuning and vocal source separation dramatically improve ASR and speaker diarization performance for low-resource Bengali speech, offering a template for other under-represented languages.
We describe our end-to-end system for Bengali long-form speech recognition (ASR) and speaker diarization submitted to the DL Sprint 4.0 competition on Kaggle. Bengali presents substantial challenges for both tasks: a large phoneme inventory, significant dialectal variation, frequent code-mixing with English, and a relative scarcity of large-scale labelled corpora. For ASR we achieve a best private Word Error Rate (WER) of 0.37738 and public WER of 0.36137, combining a BengaliAI fine-tuned Whisper medium model with Demucs source separation for vocal isolation, silence-boundary chunking, and carefully tuned generation hyperparameters. For speaker diarization we reach a best private Diarization Error Rate (DER) of 0.27671 and public DER of 0.20936 by replacing the default segmentation model inside the pyannote.audio pipeline with a Bengali-fine-tuned variant, pairing it with wespeaker-voxceleb-resnet34-LM embeddings and centroid-based agglomerative clustering. Our experiments demonstrate that domain-specific fine-tuning of the segmentation component, vocal source separation, and natural silence-aware chunking are the three most impactful design choices for low-resource Bengali speech processing.
