Approximate Kalman Filter Q-Learning for Continuous State-Space MDPs
Machine Learning
2013-09-27 v1 Machine Learning
Abstract
We seek to learn an effective policy for a Markov Decision Process (MDP) with continuous states via Q-Learning. Given a set of basis functions over state action pairs we search for a corresponding set of linear weights that minimizes the mean Bellman residual. Our algorithm uses a Kalman filter model to estimate those weights and we have developed a simpler approximate Kalman filter model that outperforms the current state of the art projected TD-Learning methods on several standard benchmark problems.
Cite
@article{arxiv.1309.6868,
title = {Approximate Kalman Filter Q-Learning for Continuous State-Space MDPs},
author = {Charles Tripp and Ross D. Shachter},
journal= {arXiv preprint arXiv:1309.6868},
year = {2013}
}
Comments
Appears in Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence (UAI2013)