Search papers, labs, and topics across Lattice.
This paper introduces a unified guidance framework that enhances Flow Matching (FM) for speech synthesis by addressing high inference latency and timbre leakage. The framework employs two strategies: Data-guidance through heterogeneous augmentation to disentangle linguistic content from acoustic residue, and Model-guidance that combines trajectory rectification with a novel intrinsic guidance objective. Experimental results show that this approach accelerates inference by nearly three times while improving speaker similarity compared to existing state-of-the-art methods.
Achieving nearly three times faster inference in speech synthesis without sacrificing speaker similarity could redefine efficiency benchmarks in the field.
Flow Matching (FM) has emerged as a powerful paradigm for speech generation but remains constrained by high inference latency and timbre leakage. To address these bottlenecks, we propose a unified guidance framework that enhances generation efficiency and robustness through two complementary strategies. On the data front, we introduce Data-guidance via heterogeneous augmentation, encouraging the model to disentangle linguistic content from acoustic residue. In parallel, we propose an enhanced Model-guidance mechanism that synergizes trajectory rectification with a novel intrinsic guidance objective. This approach distills conditional knowledge into network weights and straightens inference trajectory path, thereby eliminating Classifier-Free Guidance (CFG) overhead. Experiments demonstrate that our framework accelerates inference by nearly three times while effectively improving speaker similarity compared to state-of-the-art baselines.
