Sung-Feng Huang

This work training a LALM with reinforcement learning to reason over its own speech output generates a draft speech as a form of audio-token reasoning, critiques its own generation by reflecting on the acoustic realization in text, and produces a refined version conditioned on both the first-pass speech and the critique.

A single-layer speech enhancement model outperforms naive architectures and achieves competitive quality with a significant speedup through progressive knowledge distillation.

A single universal speech enhancement model can effectively adapt to multiple latency requirements without sacrificing performance, challenging the need for specialized models.

Text-only LLMs already contain surprisingly diverse levels of auditory knowledge, and this pre-existing knowledge strongly predicts their performance when adapted for audio-language tasks.

Time-shifted anechoic speech beats early reflections as a training target for universal speech enhancement, leading to better perceptual quality and ASR performance.