Deep Reinforcement Learning with Embedded LQR Controllers

Wouter Caarls

doi:10.1016/j.ifacol.2020.12.2261

Deep Reinforcement Learning with Embedded LQR Controllers

Robotics 2021-06-23 v1 Machine Learning Systems and Control Systems and Control

Authors: Wouter Caarls

View on arXiv ↗ PDF ↗ DOI ↗

Abstract

Reinforcement learning is a model-free optimal control method that optimizes a control policy through direct interaction with the environment. For reaching tasks that end in regulation, popular discrete-action methods are not well suited due to chattering in the goal state. We compare three different ways to solve this problem through combining reinforcement learning with classical LQR control. In particular, we introduce a method that integrates LQR control into the action set, allowing generalization and avoiding fixing the computed control in the replay memory if it is based on learned dynamics. We also embed LQR control into a continuous-action method. In all cases, we show that adding LQR control can improve performance, although the effect is more profound if it can be used to augment a discrete action set.

Keywords

reinforcement learning hierarchical reinforcement learning reinforcement learning from human feedback

Cite

@article{arxiv.2101.07175,
  title  = {Deep Reinforcement Learning with Embedded LQR Controllers},
  author = {Wouter Caarls},
  journal= {arXiv preprint arXiv:2101.07175},
  year   = {2021}
}

Comments

This work has been accepted to IFAC for publication under a Creative Commons Licence CC-BY-NC-ND

Deep Reinforcement Learning with Embedded LQR Controllers

Abstract

Keywords

Cite

Comments

Related papers