Model Predictive Control (MPC) faces computational demands and performance degradation from model inaccuracies. We propose two architectures combining Neural Network-approximated MPC (NNMPC) with Reinforcement Learning (RL). The first, Warm Start RL, initializes the RL actor with pre-trained NNMPC weights. The second, RLMPC, uses RL to generate corrective residuals for NNMPC outputs. We introduce a downsampling method reducing NNMPC input dimensions while maintaining performance. Evaluated on a rotary inverted pendulum, both architectures demonstrate runtime reductions exceeding 99% compared to traditional MPC while improving tracking performance under model uncertainties, with RL+MPC achieving 11-40% cost reduction depending on reference amplitude.
@article{arxiv.2510.03354,
title = {On Architectures for Combining Reinforcement Learning and Model Predictive Control with Runtime Improvements},
author = {Xiaolong Jia and Nikhil Bajaj},
journal= {arXiv preprint arXiv:2510.03354},
year = {2025}
}
Comments
Accepted at the 2025 IFAC Conference on Modeling, Estimation, and Control of Systems (MECC 2025), Pittsburgh, USA