Optimistic Proximal Policy Optimization

Takahisa Imagawa; Takuya Hiraoka; Yoshimasa Tsuruoka

Optimistic Proximal Policy Optimization

Machine Learning 2019-06-27 v1 Artificial Intelligence

Authors: Takahisa Imagawa , Takuya Hiraoka , Yoshimasa Tsuruoka

Abstract

Reinforcement Learning, a machine learning framework for training an autonomous agent based on rewards, has shown outstanding results in various domains. However, it is known that learning a good policy is difficult in a domain where rewards are rare. We propose a method, optimistic proximal policy optimization (OPPO) to alleviate this difficulty. OPPO considers the uncertainty of the estimated total return and optimistically evaluates the policy based on that amount. We show that OPPO outperforms the existing methods in a tabular task.

Keywords

policy gradient reinforcement learning hyperparameter optimization

Cite

@article{arxiv.1906.11075,
  title  = {Optimistic Proximal Policy Optimization},
  author = {Takahisa Imagawa and Takuya Hiraoka and Yoshimasa Tsuruoka},
  journal= {arXiv preprint arXiv:1906.11075},
  year   = {2019}
}

Comments

Exploration in RL (workshop @ ICML2019)

Optimistic Proximal Policy Optimization

Abstract

Keywords

Cite

Comments

Related papers