A Note on Optimization Formulations of Markov Decision Processes
Optimization and Control
2020-12-18 v1
Abstract
This note summarizes the optimization formulations used in the study of Markov decision processes. We consider both the discounted and undiscounted processes under the standard and the entropy-regularized settings. For each setting, we first summarize the primal, dual, and primal-dual problems of the linear programming formulation. We then detail the connections between these problems and other formulations for Markov decision processes such as the Bellman equation and the policy gradient method.
Cite
@article{arxiv.2012.09417,
title = {A Note on Optimization Formulations of Markov Decision Processes},
author = {Lexing Ying and Yuhua Zhu},
journal= {arXiv preprint arXiv:2012.09417},
year = {2020}
}