Search papers, labs, and topics across Lattice.
This study investigates the impact of disfluencies on speech translation systems, revealing that traditional models trained on cleaned text lose significant meaning by omitting these elements. The authors introduce Uh-Mazing, a benchmark featuring human-translated, disfluency-annotated Switchboard speech across eight languages, highlighting that false starts and self-repairs are the primary contributors to translation quality degradation. Notably, they demonstrate that inference-time decoding can alleviate these issues without the need for retraining the models.
Disfluencies are not just noise; they carry crucial meaning that, when ignored, significantly degrades translation quality.
Current speech translation systems, including SpeechLLMs, are trained on cleaned text and tend to strip disfluencies like filled pauses and false starts rather than translate them. We show this comes at a cost: disfluencies carry meaning that gets lost when speech is cleaned up. To study this systematically, we introduce Uh-Mazing, a benchmark of human-translated, disfluency-annotated Switchboard speech covering English into eight target languages. Across these languages and several architectures, we find that false starts and self-repairs, not filled pauses or discourse markers, drive most of the translation-quality loss, and that models which fail to preserve a disfluency tend to omit it rather than mistranslate it. We show inference-time decoding can mitigate this without retraining, and release the benchmark and code.
ETH