Bounded Exploration with World Model Uncertainty in Soft Actor-Critic Reinforcement Learning Algorithm

Ting Qiao; Henry Williams; David Valencia; Bruce MacDonald

Bounded Exploration with World Model Uncertainty in Soft Actor-Critic Reinforcement Learning Algorithm

Machine Learning 2024-12-10 v1 Systems and Control Systems and Control

Authors: Ting Qiao , Henry Williams , David Valencia , Bruce MacDonald

Abstract

One of the bottlenecks preventing Deep Reinforcement Learning algorithms (DRL) from real-world applications is how to explore the environment and collect informative transitions efficiently. The present paper describes bounded exploration, a novel exploration method that integrates both 'soft' and intrinsic motivation exploration. Bounded exploration notably improved the Soft Actor-Critic algorithm's performance and its model-based extension's converging speed. It achieved the highest score in 6 out of 8 experiments. Bounded exploration presents an alternative method to introduce intrinsic motivations to exploration when the original reward function has strict meanings.

Keywords

reinforcement learning hierarchical reinforcement learning reinforcement learning from human feedback

Cite

@article{arxiv.2412.06139,
  title  = {Bounded Exploration with World Model Uncertainty in Soft Actor-Critic Reinforcement Learning Algorithm},
  author = {Ting Qiao and Henry Williams and David Valencia and Bruce MacDonald},
  journal= {arXiv preprint arXiv:2412.06139},
  year   = {2024}
}

Comments

8 pages, 7 figures. Accepted as a poster presentation in the Australian Robotics and Automation Association (2023)

Bounded Exploration with World Model Uncertainty in Soft Actor-Critic Reinforcement Learning Algorithm

Abstract

Keywords

Cite

Comments

Related papers