Discretization error from regularized Reinforcement Learning to continuous-time stochastic control
Optimization and Control
2026-04-24 v1
Abstract
This paper establishes a rigorous connection between regularized discrete-time reinforcement learning (RL) and continuous-time stochastic optimal control. Specifically, classical RL algorithms are typically solving a regularized discrete-time Bellman equation. We study the discretization error, namely, the gap between the optimal policy induced by the regularized discrete-time Bellman equation and the true optimal feedback control of the underlying continuous-time stochastic control problem. By deriving quantitative convergence rates for this gap, we provide a rigorous foundation for understanding the stability and implementation of exploratory RL policies in stochastic continuous-time environments.
Cite
@article{arxiv.2604.21179,
title = {Discretization error from regularized Reinforcement Learning to continuous-time stochastic control},
author = {Huyên Pham and Yuming Paul Zhang and Yuhua Zhu},
journal= {arXiv preprint arXiv:2604.21179},
year = {2026}
}