EMORL-TTS: Reinforcement Learning for Fine-Grained Emotion Control in LLM-based TTS
Abstract
Recent LLM-based TTS systems achieve strong quality and zero-shot ability, but lack fine-grained emotional control due to their reliance on discrete speech tokens. Existing approaches either limit emotions to categorical labels or cannot generalize to LLM-based architectures. We propose EMORL-TTS (Fine-grained Emotion-controllable TTS with Reinforcement Learning), a framework that unifies global intensity control in the VAD space with local emphasis regulation. Our method combines supervised fine-tuning with reinforcement learning guided by task-specific rewards for emotion category, intensity, and emphasis. Moreover, we further investigate how emphasis placement modulates fine-grained emotion intensity. Experiments show that EMORL-TTS improves emotion accuracy, intensity differentiation, and emphasis clarity, while preserving synthesis quality comparable to strong LLM-based baselines.
Keywords
Cite
@article{arxiv.2510.05758,
title = {EMORL-TTS: Reinforcement Learning for Fine-Grained Emotion Control in LLM-based TTS},
author = {Haoxun Li and Yu Liu and Yuqing Sun and Hanlei Shi and Leyuan Qu and Taihao Li},
journal= {arXiv preprint arXiv:2510.05758},
year = {2026}
}
Comments
Accepted by ICASSP 2026