Autoregressive diffusion models (ARDMs) have recently been applied to speech generation, achieving state-of-the-art (SOTA) performance in zero-shot text-to-speech. By autoregressively generating continuous speech tokens with next-token diffusion, these models offer a promising alternative to next-token prediction, avoiding the technical complexities associated with discrete speech tokenization. As a relatively new paradigm, research on reinforcement learning (RL)-based fine-tuning of speech ARDMs remains limited. In this paper, we propose Autoregressive Diffusion-Direct Preference Optimization (ARDM-DPO) to advance this research. By fine-tuning the recently proposed zero-shot text-to-speech model DiTAR with DPO, we achieve significant improvements in terms of speech expressiveness and robustness for long texts.
Cite
@article{arxiv.2509.18928,
title = {Direct Preference Optimization for Speech Autoregressive Diffusion Models},
author = {Zhijun Liu and Dongya Jia and Xiaoqiang Wang and Chenpeng Du and Shuai Wang and Zhuo Chen and Haizhou Li},
journal= {arXiv preprint arXiv:2509.18928},
year = {2025}
}