Policy iteration using Q-functions: Linear dynamics with multiplicative noise
Optimization and Control
2022-12-05 v1
Abstract
This paper presents a novel model-free and fully data-driven policy iteration scheme for quadratic regulation of linear dynamics with state- and input-multiplicative noise. The implementation is similar to the least-squares temporal difference scheme for Markov decision processes, estimating Q-functions by solving a least-squares problem with instrumental variables. The scheme is compared with a model-based system identification scheme and natural policy gradient through numerical experiments.
Cite
@article{arxiv.2212.01192,
title = {Policy iteration using Q-functions: Linear dynamics with multiplicative noise},
author = {Peter Coppens and Panagiotis Patrinos},
journal= {arXiv preprint arXiv:2212.01192},
year = {2022}
}