Natural Policy Gradients In Reinforcement Learning Explained

W. J. A. van Heeswijk

Natural Policy Gradients In Reinforcement Learning Explained

Machine Learning 2022-09-07 v1 Optimization and Control

Authors: W. J. A. van Heeswijk

Abstract

Traditional policy gradient methods are fundamentally flawed. Natural gradients converge quicker and better, forming the foundation of contemporary Reinforcement Learning such as Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO). This lecture note aims to clarify the intuition behind natural policy gradients, focusing on the thought process and the key mathematical constructs.

Keywords

policy gradient reinforcement learning hyperparameter optimization

Cite

@article{arxiv.2209.01820,
  title  = {Natural Policy Gradients In Reinforcement Learning Explained},
  author = {W. J. A. van Heeswijk},
  journal= {arXiv preprint arXiv:2209.01820},
  year   = {2022}
}

Comments

14 pages, 3 figures

Related papers

View all related →