Related papers: On the value function for nonautonomous optimal co…
In this paper, we introduce Hamilton-Jacobi-Bellman (HJB) equations for Q-functions in continuous time optimal control problems with Lipschitz continuous controls. The standard Q-function used in reinforcement learning is shown to be the…
In the paper, we consider a path-dependent Hamilton-Jacobi equation with coinvariant derivatives over the space of continuous functions. Such equations arise from optimal control problems and differential games for time-delay systems. We…
We present methods for locally solving the Dynamic Programming Equations (DPE) and the Hamilton Jacobi Bellman (HJB) PDE that arise in the infinite horizon optimal control problem. The method for solving the DPE is the discrete time version…
This paper is devoted to the study of fully nonlinear stochastic Hamilton-Jacobi (HJ) equations for the optimal stochastic control problem of ordinary differential equations with random coefficients. Under the standard Lipschitz continuity…
In this paper, we study the optimal singular controls for stochastic recursive systems, in which the control has two components: the regular control, and the singular control. Under certain assumptions, we establish the dynamic programming…
We investigate the large-time behavior of the value functions of the optimal control problems on the $n$-dimensional torus which appear in the dynamic programming for the system whose states are governed by random changes. From the point of…
In this paper we prove that there exists a smooth classical solution to the HJB equation for a large class of constrained problems with utility functions that are not necessarily differentiable or strictly concave. The value function is…
We study a hybrid control system in which both discrete and continuous controls are involved. The discrete controls act on the system at a given set interface. The state of the system is changed discontinuously when the trajectory hits…
In the last decades, control problems with infinite horizons and discount factors have become increasingly central not only for economics but also for applications in artificial intelligence and machine learning. The strong links between…
We characterise the value function of the optimal dividend problem with a finite time horizon as the unique classical solution of a suitable Hamilton-Jacobi-Bellman equation. The optimal dividend strategy is realised by a Skorokhod…
In this paper, we study the delayed stochastic recursive optimal control problem with a non-Lipschitz generator, in which both the dynamics of the control system and the recursive cost functional depend on the past path segment of the state…
We study semi Lagrangian approximation schemes for Hamilton Jacobi Bellman equations arising from finite horizon optimal control problems. Classical error estimates for these schemes include the term $\frac{1}{\Delta t}$ which leads to…
This paper investigates the convergence properties of the upwind difference scheme for the Hamilton--Jacobi--Bellman (HJB) equation, a central partial differential equation in optimal control theory. First, assuming the existence of a…
In this paper, we consider the stochastic optimal control problem for jump diffusion systems with state constraints. In general, the value function of such problems is a discontinuous viscosity solution of the Hamilton-Jacobi-Bellman (HJB)…
Maximum entropy reinforcement learning (RL) methods have been successfully applied to a range of challenging sequential decision-making and control tasks. However, most of existing techniques are designed for discrete-time systems. As a…
In this manuscript, we study optimal control problems for stochastic delay differential equations using the dynamic programming approach in Hilbert spaces via viscosity solutions of the associated Hamilton-Jacobi-Bellman equations. We show…
We consider a class of diffusions controlled through the drift and jump size, and driven by a jump L\'evy process and a nondegenerate Wiener process, and we study infinite horizon (ergodic) risk-sensitive control problem for this model. We…
The numerical realization of the dynamic programming principle for continuous-time optimal control leads to nonlinear Hamilton-Jacobi-Bellman equations which require the minimization of a nonlinear mapping over the set of admissible…
An optimal control problem in the space of probability measures, and the viscosity solutions of the corresponding dynamic programming equations defined using the intrinsic linear derivative are studied. The value function is shown to be…
We study an optimal control problem in Bolza form and we consider the value function associated to this problem. We prove two verification theorems which ensure that, if a function $W$ satisfies some suitable weak continuity assumptions and…