On the Synthesis of Bellman Inequalities for Data-Driven Optimal Control
Abstract
In the context of the linear programming (LP) approach to data-driven control, one assumes that the dynamical system is unknown but can be observed indirectly through data on its evolution. Both theoretical and empirical evidence suggest that a desired suboptimality gap is often only achieved with massive exploration of the state-space. In case of linear systems, we discuss how a relatively small but sufficiently rich dataset can be exploited to generate new constraints offline and without observing the corresponding transitions. Moreover, we show how to reconstruct the associated unknown stage-costs and, when the system is stochastic, we offer insights on the related problem of estimating the expected value in the Bellman operator without re-initializing the dynamics in the same state-input pairs.
Keywords
Cite
@article{arxiv.2109.13193,
title = {On the Synthesis of Bellman Inequalities for Data-Driven Optimal Control},
author = {Andrea Martinelli and Matilde Gargiani and John Lygeros},
journal= {arXiv preprint arXiv:2109.13193},
year = {2021}
}