Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
FlowEdit introduces a lifelong adaptation framework for flow-matching text-to-speech (TTS) systems that allows for real-time pronunciation corrections without the need for model retraining. By utilizing a Modern Hopfield Network as episodic memory, FlowEdit learns and retrieves pronunciation adjustments through latent conditioning edits, optimizing token-level perturbations in the text embedding space. The method achieves a remarkable 92.7% reduction in Phoneme Error Rate for out-of-vocabulary proper nouns while preserving general-speech quality, demonstrating its effectiveness in enhancing TTS systems post-deployment.
FlowEdit slashes pronunciation errors by 92.7% in TTS systems without retraining, revolutionizing how we adapt to new vocabulary on-the-fly.
Flow-matching text-to-speech systems achieve remarkable zero-shot quality but remain static after deployment: pronunciation errors on out-of-vocabulary proper nouns persist unless the model is retrained. We introduce FlowEdit, a life-long adaptation framework for frozen flow-matching TTS that learns pronunciation corrections as latent conditioning edits rather than weight updates. When corrective feedback is provided, FlowEdit optimizes a token-level perturbation in the text embedding space, then stores the correction in a Modern Hopfield Network serving as content-addressable episodic memory. At inference, corrections are retrieved via soft attention with a similarity gate, enabling fuzzy morphological matching. On our curated benchmark of 312 multilingual proper nouns across 18 language families, FlowEdit reduces target-word Phoneme Error Rate by 92.7% relative to the zero-shot baseline while maintaining identical general-speech quality. Corrections complete in approximately 15 seconds on a single GPU.