Robust exploration in linear quadratic reinforcement learning
Optimization and Control
2019-06-05 v1 Machine Learning
Machine Learning
Abstract
This paper concerns the problem of learning control policies for an unknown linear dynamical system to minimize a quadratic cost function. We present a method, based on convex optimization, that accomplishes this task robustly: i.e., we minimize the worst-case cost, accounting for system uncertainty given the observed data. The method balances exploitation and exploration, exciting the system in such a way so as to reduce uncertainty in the model parameters to which the worst-case cost is most sensitive. Numerical simulations and application to a hardware-in-the-loop servo-mechanism demonstrate the approach, with appreciable performance and robustness gains over alternative methods observed in both.
Cite
@article{arxiv.1906.01584,
title = {Robust exploration in linear quadratic reinforcement learning},
author = {Jack Umenberger and Mina Ferizbegovic and Thomas B. Schön and Håkan Hjalmarsson},
journal= {arXiv preprint arXiv:1906.01584},
year = {2019}
}