Jun Suzuki

Forget catastrophic forgetting: inserting speech-specific layers into frozen text LLMs lets you build strong speech models without trashing your original text smarts.

Ditch the learning rate decay: LLMs pre-trained with a stable learning rate after warmup show superior adaptability and downstream performance after supervised fine-tuning compared to those trained with decay-based schedules.