Search papers, labs, and topics across Lattice.
This study evaluates the performance of compact, domain-specialized automatic speech recognition (ASR) models against large multilingual foundation models for conversational speech in 19 African languages using the WAXAL corpus. Fine-tuned edge models achieved a macro-averaged word error rate (WER) of 38.0%, significantly outperforming the best zero-shot baseline at 64.9%, demonstrating that domain specialization is more effective than model scale for spontaneous African speech. Additionally, the research provides a linguistically-grounded error taxonomy and highlights the limitations of WER as a sole performance metric, particularly for syllabary-script languages, while releasing valuable resources for future ASR research in Africa.
Compact, fine-tuned ASR models can outperform massive multilingual models by 26.9 percentage points in recognizing spontaneous African speech.
We evaluate whether compact domain-specialized ASR models can outperform massively multilingual foundation models for conversational African speech across 19 languages in the WAXAL corpus. Fine-tuned edge models achieve a macro-averaged WER of $38.0\%$ compared to $64.9\%$ for the best zero-shot baseline, a $26.9$ percentage-point reduction using models $3-40\times$ smaller. Results confirm that domain specialization dominates scale for spontaneous African speech. Cross-domain evaluation shows that fine-tuned models recover usable performance on out-of-distribution (OOD) speech, while zero-shot models regain an advantage when the test domain matches their pretraining distribution. A distributed native-speaker audit across all surveyed languages produces a linguistically-grounded error taxonomy, showing that CTC and autoregressive architectures behave differently across language families. We further show that WER alone misrepresents performance for syllabary-script languages where CER/WER ratios reveal substantially higher character-level accuracy than headline WER suggests. Finally, to contribute to future African ASR research, we release all model weights, fine-tuning and evaluation scripts, and a cleaned WAXAL subset covering all $19$ languages.
