Ho-Lam Chung

This work training a LALM with reinforcement learning to reason over its own speech output generates a draft speech as a form of audio-token reasoning, critiques its own generation by reflecting on the acoustic realization in text, and produces a refined version conditioned on both the first-pass speech and the critique.

ORCA not only boosts performance by 26.4 points but also restores critical speaker identity cues that traditional models overlook.

SR-FD reduces word error rates by over a third, transforming the landscape of intelligibility in few-step TTS synthesis.

Taiwanese Mandarin TTS systems can achieve a 63.9% reduction in word error rates by using a context-adapted tokenizer and language model.

LatentASR transforms frozen ASR models into adaptive systems that intelligently allocate compute resources, achieving significant WER reductions on challenging inputs without the need for extensive retraining.

Leveraging lecture context can boost technical term recognition in Mandarin ASR systems by over 15% without sacrificing overall accuracy.

Bridging the gap between audio reconstruction and language modeling objectives yields neural audio codecs that are both more acoustically faithful and linguistically predictable.

Overcome LALM's struggles with localized dialectal prosody: a new Taiwanese audio-text dataset and fine-tuning strategy boosts accuracy by 6.5% on the TAU Benchmark.