A Sliding-Window Algorithm for Markov Decision Processes with Arbitrarily Changing Rewards and Transitions
Machine Learning
2018-05-28 v1 Machine Learning
Abstract
We consider reinforcement learning in changing Markov Decision Processes where both the state-transition probabilities and the reward functions may vary over time. For this problem setting, we propose an algorithm using a sliding window approach and provide performance guarantees for the regret evaluated against the optimal non-stationary policy. We also characterize the optimal window size suitable for our algorithm. These results are complemented by a sample complexity bound on the number of sub-optimal steps taken by the algorithm. Finally, we present some experimental results to support our theoretical analysis.
Cite
@article{arxiv.1805.10066,
title = {A Sliding-Window Algorithm for Markov Decision Processes with Arbitrarily Changing Rewards and Transitions},
author = {Pratik Gajane and Ronald Ortner and Peter Auer},
journal= {arXiv preprint arXiv:1805.10066},
year = {2018}
}