Probabilistic design of optimal sequential decision-making algorithms in learning and control
Abstract
This survey is focused on certain sequential decision-making problems that involve optimizing over probability functions. We discuss the relevance of these problems for learning and control. The survey is organized around a framework that combines a problem formulation and a set of resolution methods. The formulation consists of an infinite-dimensional optimization problem. The methods come from approaches to search optimal solutions in the space of probability functions. Through the lenses of this overarching framework we revisit popular learning and control algorithms, showing that these naturally arise from suitable variations on the formulation mixed with different resolution methods. A running example, for which we make the code available, complements the survey. Finally, a number of challenges arising from the survey are also outlined.
Cite
@article{arxiv.2201.05212,
title = {Probabilistic design of optimal sequential decision-making algorithms in learning and control},
author = {Emiland Garrabe and Giovanni Russo},
journal= {arXiv preprint arXiv:2201.05212},
year = {2023}
}
Comments
This is an authors' version of the work that is published in Annual Reviews in Control, Vol. 54, 2022, Pages 81-102. Changes were made to this version by the publisher prior to publication. The final version of record is available at https://doi.org/10.1016/j.arcontrol.2022.09.003