Riemannian Proximal Policy Optimization
Machine Learning
2020-05-20 v1 Machine Learning
Abstract
In this paper, We propose a general Riemannian proximal optimization algorithm with guaranteed convergence to solve Markov decision process (MDP) problems. To model policy functions in MDP, we employ Gaussian mixture model (GMM) and formulate it as a nonconvex optimization problem in the Riemannian space of positive semidefinite matrices. For two given policy functions, we also provide its lower bound on policy improvement by using bounds derived from the Wasserstein distance of GMMs. Preliminary experiments show the efficacy of our proposed Riemannian proximal policy optimization algorithm.
Cite
@article{arxiv.2005.09195,
title = {Riemannian Proximal Policy Optimization},
author = {Shijun Wang and Baocheng Zhu and Chen Li and Mingzhe Wu and James Zhang and Wei Chu and Yuan Qi},
journal= {arXiv preprint arXiv:2005.09195},
year = {2020}
}
Comments
12 pages, 1 figures