Taylor Expansion Policy Optimization

Yunhao Tang; Michal Valko; Rémi Munos

Taylor Expansion Policy Optimization

Machine Learning 2020-03-16 v1 Machine Learning

Authors: Yunhao Tang , Michal Valko , Rémi Munos

Abstract

In this work, we investigate the application of Taylor expansions in reinforcement learning. In particular, we propose Taylor expansion policy optimization, a policy optimization formalism that generalizes prior work (e.g., TRPO) as a first-order special case. We also show that Taylor expansions intimately relate to off-policy evaluation. Finally, we show that this new formulation entails modifications which improve the performance of several state-of-the-art distributed algorithms.

Keywords

policy gradient reinforcement learning first-order logic

Cite

@article{arxiv.2003.06259,
  title  = {Taylor Expansion Policy Optimization},
  author = {Yunhao Tang and Michal Valko and Rémi Munos},
  journal= {arXiv preprint arXiv:2003.06259},
  year   = {2020}
}

Related papers

View all related →