A Model-Based Approach for Improving Reinforcement Learning Efficiency Leveraging Expert Observations
Machine Learning
2025-02-28 v1
Abstract
This paper investigates how to incorporate expert observations (without explicit information on expert actions) into a deep reinforcement learning setting to improve sample efficiency. First, we formulate an augmented policy loss combining a maximum entropy reinforcement learning objective with a behavioral cloning loss that leverages a forward dynamics model. Then, we propose an algorithm that automatically adjusts the weights of each component in the augmented loss function. Experiments on a variety of continuous control tasks demonstrate that the proposed algorithm outperforms various benchmarks by effectively utilizing available expert observations.
Cite
@article{arxiv.2402.18836,
title = {A Model-Based Approach for Improving Reinforcement Learning Efficiency Leveraging Expert Observations},
author = {Erhan Can Ozcan and Vittorio Giammarino and James Queeney and Ioannis Ch. Paschalidis},
journal= {arXiv preprint arXiv:2402.18836},
year = {2025}
}