Recent advances in text-to-speech (TTS) synthesis, particularly those leveraging large language models (LLMs), have significantly improved expressiveness and naturalness. However, generating human-like, interactive dialogue speech remains challenging. Current systems face limitations due to the scarcity of dual-track data and difficulties in achieving naturalness, contextual coherence, and interactional dynamics, such as turn-taking, overlapping speech, and speaker consistency, in multi-turn conversations. To address these challenges, we propose DialoSpeech, a dual-track architecture combining a large language model with Chunked Flow Matching for expressive, human-like dialogue speech synthesis. DialoSpeech generates natural multi-turn conversations with coherent speaker turns and natural overlaps, supporting both Chinese and English and cross-lingual speech synthesis. We introduce a data processing pipeline to construct dual-track dialogue datasets, facilitating scalable training and experimental validation. Experiments show that our model outperforms baselines, offering a solution for generating human-like spoken dialogues. Audio samples are available at https://tiamojames.github.io/DialoSpeech
@article{arxiv.2510.08373,
title = {DialoSpeech: Dual-Speaker Dialogue Generation with LLM and Flow Matching},
author = {Hanke Xie and Dake Guo and Chengyou Wang and Yue Li and Wenjie Tian and Xinfa Zhu and Xinsheng Wang and Xiulin Li and Guanqiong Miao and Bo Liu and Lei Xie},
journal= {arXiv preprint arXiv:2510.08373},
year = {2025}
}