Metrics for Markov Decision Processes with Infinite State Spaces
Artificial Intelligence
2012-07-09 v1
Abstract
We present metrics for measuring state similarity in Markov decision processes (MDPs) with infinitely many states, including MDPs with continuous state spaces. Such metrics provide a stable quantitative analogue of the notion of bisimulation for MDPs, and are suitable for use in MDP approximation. We show that the optimal value function associated with a discounted infinite horizon planning task varies continuously with respect to our metric distances.
Cite
@article{arxiv.1207.1386,
title = {Metrics for Markov Decision Processes with Infinite State Spaces},
author = {Norman Ferns and Prakash Panangaden and Doina Precup},
journal= {arXiv preprint arXiv:1207.1386},
year = {2012}
}
Comments
Appears in Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence (UAI2005)