Search papers, labs, and topics across Lattice.
This paper investigates the challenge of improving dialect recognition in automatic speech recognition (ASR) systems while maintaining high accuracy for Mandarin. By employing an adaptation pipeline that combines continual pre-training, dialect supervised fine-tuning, and On-Policy Self-Distillation (OPSD), the authors successfully refine dialect recognition without compromising Mandarin performance. The key result shows that OPSD enhances dialect recognition accuracy without increasing character error rates (CER) for Mandarin, unlike traditional teacher-forced fine-tuning methods.
Dialect recognition in ASR can be significantly improved without sacrificing Mandarin accuracy, thanks to a novel self-distillation approach.
Recent large-scale ASR models already achieve strong Mandarin recognition accuracy and have some ability to recognize Chinese dialects. However, their dialect recognition accuracy is still limited in real-world speech. Direct dialect adaptation can lower dialect CER, but it may also raise Mandarin CER. We therefore study how to adapt a capable ASR model to improve multi-dialect recognition without degrading Mandarin recognition. We adopt an adaptation pipeline where continual pre-training (CPT) and dialect supervised fine-tuning (SFT) provide a strong foundation, and On-Policy Self-Distillation (OPSD) serves as the final refinement. OPSD addresses the train--test mismatch in autoregressive ASR by training the student model on its own decoded prefixes while a frozen teacher, conditioned on the reference transcript as privileged context, provides soft token-level targets. This replaces hard cross-entropy updates on dialect data with distillation, preserving Mandarin ability while refining dialect recognition. We instantiate the framework with Qwen3-ASR-1.7B and evaluate it on public and internal Mandarin and dialect test sets. Under matched refinement data and schedule, OPSD improves dialect recognition without raising Mandarin CER, whereas continued teacher-forced fine-tuning increases Mandarin CER. We will release the model weights and evaluation scripts.
