Model-based Reinforcement Learning for Parameterized Action Spaces

Renhao Zhang; Haotian Fu; Yilin Miao; George Konidaris

Model-based Reinforcement Learning for Parameterized Action Spaces

Machine Learning 2024-05-27 v3 Artificial Intelligence

Authors: Renhao Zhang , Haotian Fu , Yilin Miao , George Konidaris

Abstract

We propose a novel model-based reinforcement learning algorithm -- Dynamics Learning and predictive control with Parameterized Actions (DLPA) -- for Parameterized Action Markov Decision Processes (PAMDPs). The agent learns a parameterized-action-conditioned dynamics model and plans with a modified Model Predictive Path Integral control. We theoretically quantify the difference between the generated trajectory and the optimal trajectory during planning in terms of the value they achieved through the lens of Lipschitz Continuity. Our empirical results on several standard benchmarks show that our algorithm achieves superior sample efficiency and asymptotic performance than state-of-the-art PAMDP methods.

Keywords

reinforcement learning markov decision processes dynamical systems

Cite

@article{arxiv.2404.03037,
  title  = {Model-based Reinforcement Learning for Parameterized Action Spaces},
  author = {Renhao Zhang and Haotian Fu and Yilin Miao and George Konidaris},
  journal= {arXiv preprint arXiv:2404.03037},
  year   = {2024}
}

Related papers

View all related →