Bootstrap Policy Iteration for Stochastic LQ Tracking with Multiplicative Noise
Abstract
This paper studies the optimal tracking control problem for continuous-time stochastic linear systems with multiplicative noise. The solution framework involves solving a stochastic algebraic Riccati equation for the feedback gain and a Sylvester equation for the feedforward gain. To enable model-free optimal tracking, we first develop a two-phase bootstrap policy iteration (B-PI) algorithm, which bootstraps a stabilizing control gain from the trivially initialized zero-value start and proceeds with standard policy iteration. Building on this algorithm, we propose a data-driven, off-policy reinforcement learning approach that ensures convergence to the optimal feedback gain under the interval excitation condition. We further introduce a data-driven method to compute the feedforward using the obtained feedback gain. Additionally, for systems with state-dependent noise, we propose a shadow system-based optimal tracking method to eliminate the need for probing noise. The effectiveness of the proposed methods is demonstrated through numerical examples.
Cite
@article{arxiv.2508.20394,
title = {Bootstrap Policy Iteration for Stochastic LQ Tracking with Multiplicative Noise},
author = {Jiayu Chen and Zhenhui Xu and Xinghu Wang},
journal= {arXiv preprint arXiv:2508.20394},
year = {2025}
}