Stochastic Optimal Control via Hilbert Space Embeddings of Distributions
Abstract
Kernel embeddings of distributions have recently gained significant attention in the machine learning community as a data-driven technique for representing probability distributions. Broadly, these techniques enable efficient computation of expectations by representing integral operators as elements in a reproducing kernel Hilbert space. We apply these techniques to the area of stochastic optimal control theory and present a method to compute approximately optimal policies for stochastic systems with arbitrary disturbances. Our approach reduces the optimization problem to a linear program, which can easily be solved via the Lagrangian dual, without resorting to gradient-based optimization algorithms. We focus on discrete-time dynamic programming, and demonstrate our proposed approach on a linear regulation problem, and on a nonlinear target tracking problem. This approach is broadly applicable to a wide variety of optimal control problems, and provides a means of working with stochastic systems in a data-driven setting.
Cite
@article{arxiv.2103.12759,
title = {Stochastic Optimal Control via Hilbert Space Embeddings of Distributions},
author = {Adam J. Thorpe and Meeko M. K. Oishi},
journal= {arXiv preprint arXiv:2103.12759},
year = {2021}
}