Stochastic Optimal Control with Side Information and Bayesian Learning
Optimization and Control
2026-02-26 v1 Statistics Theory
Statistics Theory
Abstract
We study infinite-horizon stochastic optimal control problems with observable side information: a Markov chain that modulates an unknown context-conditional randomness distribution. Since this distribution is unknown, we propose a Bayesian reformulation based on a parametric density model and posterior predictive dynamics, which yields a Bayesian Bellman equation. We prove posterior consistency under Markov samples and, under correct specification and identifiability, uniform convergence of the Bayesian value function. Finally, we establish Bernstein--von Mises-type asymptotic normality for the data-driven contextual optimal value.
Cite
@article{arxiv.2602.22047,
title = {Stochastic Optimal Control with Side Information and Bayesian Learning},
author = {Johannes Milz and Alexander Shapiro and Enlu Zhou},
journal= {arXiv preprint arXiv:2602.22047},
year = {2026}
}