Related papers: A Temporal Difference Method for Stochastic Contin…
The Bellman equation and its continuous form, the Hamilton-Jacobi-Bellman equation, are ubiquitous in reinforcement learning and control theory. However, these equations become intractable for high-dimensional or nonlinear systems. This…
In this paper, we study a time-inconsistent stochastic optimal control problem with a recursive cost functional by a multi-person hierarchical differential game approach. An equilibrium strategy of this problem is constructed and a…
Reinforcement Learning (RL) remains a central optimisation framework in machine learning. Although RL agents can converge to optimal solutions, the definition of ``optimality'' depends on the environment's statistical properties. The…
Many optimal control problems are formulated as two point boundary value problems (TPBVPs) with conditions of optimality derived from the Hamilton-Jacobi-Bellman (HJB) equations. In most cases, it is challenging to solve HJBs due to the…
In this paper, we first establish the dynamic programming principle for stochastic optimal control problems defined on compact Riemannian manifolds without boundary. Subsequently, we derive the associated Hamilton-Jacobi-Bellman (HJB)…
We consider the problem of time-optimal path planning for simple nonholonomic vehicles. In previous similar work, the vehicle has been simplified to a point mass and the obstacles have been stationary. Our formulation accounts for a…
We study the problem of optimal portfolio selection under stochastic volatility within a continuous time reinforcement learning framework with portfolio constraints. Exploration is modeled through entropy-regularized relaxed controls, where…
This paper proposes a new framework to model control systems in which a dynamic friction occurs. The model consists in a controlled differential inclusion with a discontinuous right hand side, which still preserves existence and uniqueness…
In this paper, we propose Q-learning algorithms for continuous-time deterministic optimal control problems with Lipschitz continuous controls. Our method is based on a new class of Hamilton-Jacobi-Bellman (HJB) equations derived from…
This paper establishes a rigorous connection between regularized discrete-time reinforcement learning (RL) and continuous-time stochastic optimal control. Specifically, classical RL algorithms are typically solving a regularized…
Hard constraints in reinforcement learning (RL) often degrade policy performance. Lagrangian methods offer a way to blend objectives with constraints, but require intricate reward engineering and parameter tuning. In this work, we extend…
Many real-world control problems, ranging from finance to robotics, evolve in continuous time with non-uniform, event-driven decisions. Standard discrete-time reinforcement learning (RL), based on fixed-step Bellman updates, struggles in…
An optimal control problem is considered for a stochastic differential equation with the cost functional determined by a backward stochastic Volterra integral equation (BSVIE, for short). This kind of cost functional can cover the general…
This paper addresses the numerical solution of backward stochastic differential equations (BSDEs) arising in stochastic optimal control. Specifically, we investigate two BSDEs: one derived from the Hamilton-Jacobi-Bellman equation and the…
A general time-inconsistent optimal control problem is considered for stochastic differential equations with deterministic coefficients. Under suitable conditions, a Hamilton-Jacobi-Bellman type equation is derived for the equilibrium value…
We treat infinite horizon optimal control problems by solving the associated stationary Hamilton-Jacobi-Bellman (HJB) equation numerically to compute the value function and an optimal feedback law. The dynamical systems under consideration…
We consider a stochastic optimal control problem where the controller can anticipate the evolution of the driving noise over some dynamically changing time window. The controlled state dynamics are understood as a rough differential…
In optimal control problems of control-affine systems, whose solutions are bang-bang or singular type, verification of optimality using the Hamilton-Jacobi-Bellman (HJB) equation involves the computation of partial derivatives of switching…
We study the problem of generating control laws for systems with unknown dynamics. Our approach is to represent the controller and the value function with neural networks, and to train them using loss functions adapted from the…
To investigate a time-consistent optimal strategy for the continuous time mean-variance model, we develop a new method to establish the Bellman principle. Based on this new method, we obtain a time-consistent dynamic optimal strategy that…