Multi-Agent Reinforcement Learning via Distributed MPC as a Function Approximator
Abstract
This paper presents a novel approach to multi-agent reinforcement learning (RL) for linear systems with convex polytopic constraints. Existing work on RL has demonstrated the use of model predictive control (MPC) as a function approximator for the policy and value functions. The current paper is the first work to extend this idea to the multi-agent setting. We propose the use of a distributed MPC scheme as a function approximator, with a structure allowing for distributed learning and deployment. We then show that Q-learning updates can be performed distributively without introducing nonstationarity, by reconstructing a centralized learning update. The effectiveness of the approach is demonstrated on two numerical examples.
Cite
@article{arxiv.2312.05166,
title = {Multi-Agent Reinforcement Learning via Distributed MPC as a Function Approximator},
author = {Samuel Mallick and Filippo Airaldi and Azita Dabiri and Bart De Schutter},
journal= {arXiv preprint arXiv:2312.05166},
year = {2025}
}
Comments
12 pages, 8 figures, accepted for publication in Automatica, code can be found at https://github.com/SamuelMallick/dmpcrl-concept/tree/paper-2023