A Convergence Result for Regularized Actor-Critic Methods
Machine Learning
2019-10-23 v2 Machine Learning
Abstract
In this paper, we present a probability one convergence proof, under suitable conditions, of a certain class of actor-critic algorithms for finding approximate solutions to entropy-regularized MDPs using the machinery of stochastic approximation. To obtain this overall result, we prove the convergence of policy evaluation with general regularizers when using linear approximation architectures and show convergence of entropy-regularized policy improvement.
Keywords
Cite
@article{arxiv.1907.06138,
title = {A Convergence Result for Regularized Actor-Critic Methods},
author = {Wesley Suttle and Zhuoran Yang and Kaiqing Zhang and Ji Liu},
journal= {arXiv preprint arXiv:1907.06138},
year = {2019}
}