A Demonstration of Issues with Value-Based Multiobjective Reinforcement Learning Under Stochastic State Transitions
Machine Learning
2021-03-16 v1 Multiagent Systems
Machine Learning
Abstract
We report a previously unidentified issue with model-free, value-based approaches to multiobjective reinforcement learning in the context of environments with stochastic state transitions. An example multiobjective Markov Decision Process (MOMDP) is used to demonstrate that under such conditions these approaches may be unable to discover the policy which maximises the Scalarised Expected Return, and in fact may converge to a Pareto-dominated solution. We discuss several alternative methods which may be more suitable for maximising SER in MOMDPs with stochastic transitions.
Cite
@article{arxiv.2004.06277,
title = {A Demonstration of Issues with Value-Based Multiobjective Reinforcement Learning Under Stochastic State Transitions},
author = {Peter Vamplew and Cameron Foale and Richard Dazeley},
journal= {arXiv preprint arXiv:2004.06277},
year = {2021}
}
Comments
6 pages. Accepted for presentation in the Adaptive and Learning Agents Workshop, AAMAS 2020