Behavior Constraining in Weight Space for Offline Reinforcement Learning

Phillip Swazinna; Steffen Udluft; Daniel Hein; Thomas Runkler

Behavior Constraining in Weight Space for Offline Reinforcement Learning

Machine Learning 2021-07-13 v1

Authors: Phillip Swazinna , Steffen Udluft , Daniel Hein , Thomas Runkler

Abstract

In offline reinforcement learning, a policy needs to be learned from a single pre-collected dataset. Typically, policies are thus regularized during training to behave similarly to the data generating policy, by adding a penalty based on a divergence between action distributions of generating and trained policy. We propose a new algorithm, which constrains the policy directly in its weight space instead, and demonstrate its effectiveness in experiments.

Keywords

reinforcement learning machine learning theory imitation learning

Cite

@article{arxiv.2107.05479,
  title  = {Behavior Constraining in Weight Space for Offline Reinforcement Learning},
  author = {Phillip Swazinna and Steffen Udluft and Daniel Hein and Thomas Runkler},
  journal= {arXiv preprint arXiv:2107.05479},
  year   = {2021}
}

Comments

Accepted at ESANN 2021

Related papers

View all related →