Related papers: Hamilton-Jacobi-Bellman Equations for Q-Learning i…
In this paper, we propose Q-learning algorithms for continuous-time deterministic optimal control problems with Lipschitz continuous controls. Our method is based on a new class of Hamilton-Jacobi-Bellman (HJB) equations derived from…
We address the crucial yet underexplored stability properties of the Hamilton--Jacobi--Bellman (HJB) equation in model-free reinforcement learning contexts, specifically for Lipschitz continuous optimal control problems. We bridge the gap…
Maximum entropy reinforcement learning (RL) methods have been successfully applied to a range of challenging sequential decision-making and control tasks. However, most of existing techniques are designed for discrete-time systems. As a…
This paper proposes a new framework to model control systems in which a dynamic friction occurs. The model consists in a controlled differential inclusion with a discontinuous right hand side, which still preserves existence and uniqueness…
The Bellman equation and its continuous-time counterpart, the Hamilton-Jacobi-Bellman (HJB) equation, serve as necessary conditions for optimality in reinforcement learning and optimal control. While the value function is known to be the…
Continuous-time reinforcement learning offers an appealing formalism for describing control problems in which the passage of time is not naturally divided into discrete increments. Here we consider the problem of predicting the distribution…
In this paper we study the fully nonlinear stochastic Hamilton-Jacobi-Bellman (HJB) equation for the optimal stochastic control problem of stochastic differential equations with random coefficients. The notion of viscosity solution is…
In this paper, we present a novel algorithm named synchronous integral Q-learning, which is based on synchronous policy iteration, to solve the continuous-time infinite horizon optimal control problems of input-affine system dynamics. The…
Many optimal control problems are formulated as two point boundary value problems (TPBVPs) with conditions of optimality derived from the Hamilton-Jacobi-Bellman (HJB) equations. In most cases, it is challenging to solve HJBs due to the…
We present a new formulation for the computation of solutions of a class of Hamilton Jacobi Bellman (HJB) equations on closed smooth surfaces of co-dimension one. For the class of equations considered in this paper, the viscosity solution…
In this article, a class of optimal control problems of differential equations with delays are investigated for which the associated Hamilton-Jacobi-Bellman (HJB) equations are nonlinear partial differential equations with delays. This type…
We study a class of optimal control problems with state constraints where the state equation is a differential equation with delays. This class includes some problems arising in economics, in particular the so-called models with time to…
We study the exploratory Hamilton--Jacobi--Bellman (HJB) equation arising from the entropy-regularized exploratory control problem, which was formulated by Wang, Zariphopoulou and Zhou (J. Mach. Learn. Res., 21, 2020) in the context of…
Convex Q-learning is a recent approach to reinforcement learning, motivated by the possibility of a firmer theory for convergence, and the possibility of making use of greater a priori knowledge regarding policy or value function structure.…
The purpose of this paper is to describe the numerical solution of the Hamilton-Jacobi-Bellman (HJB) for an optimal control problem for quantum spin systems. This HJB equation is a first order nonlinear partial differential equation defined…
Optimal control and the associated second-order Hamilton-Jacobi-Bellman (HJB) equation are studied for unbounded stochastic evolution systems in Hilbert spaces. A new notion of viscosity solution, featured by absence of B-continuity, is…
Computing optimal feedback controls for nonlinear systems generally requires solving Hamilton-Jacobi-Bellman (HJB) equations, which are notoriously difficult when the state dimension is large. Existing strategies for high-dimensional…
In this article, a notion of viscosity solutions is introduced for second order path-dependent Hamilton-Jacobi-Bellman (PHJB) equations associated with optimal control problems for path-dependent stochastic differential equations. We…
In this article, a notion of viscosity solutions is introduced for second order path-dependent Hamilton-Jacobi-Bellman (PHJB) equations associated with optimal control problems for path-dependent stochastic evolution equations in Hilbert…
In this article, a notion of viscosity solutions is introduced for first order path-dependent Hamilton-Jacobi-Bellman (PHJB) equations associated with optimal control problems for path-dependent evolution equations in Hilbert space. We…