Dialogue assistants are rapidly becoming an indispensable daily aid. To avoid the significant effort needed to hand-craft the required dialogue flow, the Dialogue Management (DM) module can be cast as a continuous Markov Decision Process (MDP) and trained through Reinforcement Learning (RL). Several RL models have been investigated over recent years. However, the lack of a common benchmarking framework makes it difficult to perform a fair comparison between different models and their capability to generalise to different environments. Therefore, this paper proposes a set of challenging simulated environments for dialogue model development and evaluation. To provide some baselines, we investigate a number of representative parametric algorithms, namely deep reinforcement learning algorithms - DQN, A2C and Natural Actor-Critic and compare them to a non-parametric model, GP-SARSA. Both the environments and policy models are implemented using the publicly available PyDial toolkit and released on-line, in order to establish a testbed framework for further experiments and to facilitate experimental reproducibility.
@article{arxiv.1711.11023,
title = {A Benchmarking Environment for Reinforcement Learning Based Task Oriented Dialogue Management},
author = {Iñigo Casanueva and Paweł Budzianowski and Pei-Hao Su and Nikola Mrkšić and Tsung-Hsien Wen and Stefan Ultes and Lina Rojas-Barahona and Steve Young and Milica Gašić},
journal= {arXiv preprint arXiv:1711.11023},
year = {2018}
}
Comments
Accepted at the Deep Reinforcement Learning Symposium, 31st Conference on Neural Information Processing Systems (NIPS 2017) Paper updated with minor changes