English

Enhancing Flow Matching with A Unified Guidance Framework for Efficient and Robust Speech Synthesis

Sound 2026-07-01 v1 Artificial Intelligence

Abstract

Flow Matching (FM) has emerged as a powerful paradigm for speech generation but remains constrained by high inference latency and timbre leakage. To address these bottlenecks, we propose a unified guidance framework that enhances generation efficiency and robustness through two complementary strategies. On the data front, we introduce Data-guidance via heterogeneous augmentation, encouraging the model to disentangle linguistic content from acoustic residue. In parallel, we propose an enhanced Model-guidance mechanism that synergizes trajectory rectification with a novel intrinsic guidance objective. This approach distills conditional knowledge into network weights and straightens inference trajectory path, thereby eliminating Classifier-Free Guidance (CFG) overhead. Experiments demonstrate that our framework accelerates inference by nearly three times while effectively improving speaker similarity compared to state-of-the-art baselines.

Keywords

Cite

@article{arxiv.2607.00363,
  title  = {Enhancing Flow Matching with A Unified Guidance Framework for Efficient and Robust Speech Synthesis},
  author = {Zuda Yu and Qianhui Xu and Ting Chen and Junhui Zhang and Tao Fu and Hongjiang Yu and Qiangqing Wang and Yang Song},
  journal= {arXiv preprint arXiv:2607.00363},
  year   = {2026}
}

Comments

Accepted to INTERSPEECH 2026