An Efficient Solution to s-Rectangular Robust Markov Decision Processes
Machine Learning
2023-02-01 v1 Optimization and Control
Abstract
We present an efficient robust value iteration for \texttt{s}-rectangular robust Markov Decision Processes (MDPs) with a time complexity comparable to standard (non-robust) MDPs which is significantly faster than any existing method. We do so by deriving the optimal robust Bellman operator in concrete forms using our water filling lemma. We unveil the exact form of the optimal policies, which turn out to be novel threshold policies with the probability of playing an action proportional to its advantage.
Cite
@article{arxiv.2301.13642,
title = {An Efficient Solution to s-Rectangular Robust Markov Decision Processes},
author = {Navdeep Kumar and Kfir Levy and Kaixin Wang and Shie Mannor},
journal= {arXiv preprint arXiv:2301.13642},
year = {2023}
}
Comments
arXiv admin note: substantial text overlap with arXiv:2205.14327