Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the issue of uncontrolled transcription style in automatic speech recognition (ASR) models, which leads to significant decoding instability and evaluation confounding. By employing coverage-aware decoder task tokens trained on parallel verbatim and intended transcript pairs, the authors achieve a remarkable improvement in disfluency detection, raising the F1 score from 10% to 79% in zero-shot settings, even with English-only training. Additionally, they introduce a novel task called "verbatimize" for scalable creation of high-quality verbatim transcriptions, while also enhancing word-level timing accuracy beyond traditional forced-alignment methods.
Achieving a 79% F1 score in disfluency detection with zero-shot learning demonstrates that ASR models can effectively control transcription style without extensive retraining.
Modern ASR models trained on heterogeneously annotated data treat transcription style (verbatim vs. intended) as an uncontrolled latent variable, causing measurable decoding instability, evaluation confounding (up to 60% of reported WER attributable to style mismatch), and unreliable word-level timing. We show that models already encode both styles; the challenge is controlled activation. Using coverage-aware decoder task tokens trained on parallel verbatim/intended transcript pairs, we raise German disfluency F1 from 10% to 79% zero-shot, despite English-only training. Full English-only fine-tuning surpasses all baselines in verbatim accuracy, disfluency detection, and intended-mode quality across both languages. We further introduce supervised cross-attention fine-tuning that improves word-level timestamps on disfluent speech beyond forced-alignment baselines. Finally, we propose verbatimize, a new task enabling scalable creation and enrichment of speech corpora with high-quality canonical verbatim transcriptions.