Related papers: Is Bellman Equation Enough for Learning Control?
In this paper, we study one kind of stochastic recursive optimal control problem with the obstacle constraints for the cost function where the cost function is described by the solution of one reflected backward stochastic differential…
The numerical realization of the dynamic programming principle for continuous-time optimal control leads to nonlinear Hamilton-Jacobi-Bellman equations which require the minimization of a nonlinear mapping over the set of admissible…
We study semi Lagrangian approximation schemes for Hamilton Jacobi Bellman equations arising from finite horizon optimal control problems. Classical error estimates for these schemes include the term $\frac{1}{\Delta t}$ which leads to…
In this paper, we consider discrete-time infinite horizon problems of optimal control to a terminal set of states. These are the problems that are often taken as the starting point for adaptive dynamic programming. Under very general…
Empowerment is an information-theoretic method that can be used to intrinsically motivate learning agents. It attempts to maximize an agent's control over the environment by encouraging visiting states with a large number of reachable next…
H{\infty} control of nonlinear continuous-time system depends on the solution of the Hamilton-Jacobi-Isaacs (HJI) equation, which has been proved impossible to obtain a closed-form solution due to the nonlinearity of HJI equation. In order…
Stochastic optimal control problems governed by delay equations with delay in the control are usually more difficult to study than the the ones when the delay appears only in the state. This is particularly true when we look at the…
The long-time average behavior of the value function in the calculus of variations is known to be connected to the existence of the limit of the corresponding Abel means. Still in the Tonelli case, such a limit is in turn related to the…
The convergence of many reinforcement learning (RL) algorithms with linear function approximation has been investigated extensively but most proofs assume that these methods converge to a unique solution. In this paper, we provide a…
This work proposes an optimal safe controller minimizing an infinite horizon cost functional subject to control barrier functions (CBFs) safety conditions. The constrained optimal control problem is reformulated as a minimization problem of…
In this article, a notion of viscosity solutions is introduced for second order path-dependent Hamilton-Jacobi-Bellman (PHJB) equations associated with optimal control problems for path-dependent stochastic evolution equations in Hilbert…
In this paper we study an optimization problem in which the control is information, more precisely, the control is a $\sigma$-algebra or a filtration. In a dynamic setting, we establish the dynamic programming principle and the law…
We consider mean field social optimization in nonlinear diffusion models. By dynamic programming with a representative agent employing cooperative optimizer selection, we derive a new Hamilton--Jacobi--Bellman (HJB) equation to be called…
We study a finite horizon optimal control problem for the continuity equation under a weighted integral state constraint on the mass outside a fixed set. The model is cast in a Hilbert framework for densities. On a suitable invariant…
Continuous-time stochastic processes underlie many natural and engineered systems. In healthcare, autonomous driving, and industrial control, direct interaction with the environment is often unsafe or impractical, motivating offline…
The main goal of this paper is to establish existence, regularity and uniqueness results for the solution of a Hamilton-Jacobi-Bellman (HJB) equation, whose operator is an elliptic integro-differential operator. The HJB equation studied in…
Following the recent resurgence in establishing linear control theoretic benchmarks for reinforcement leaning (RL)-based policy optimization (PO) for complex dynamical systems with continuous state and action spaces, an optimal control…
Reinforcement learning (RL) is promising for complicated stochastic nonlinear control problems. Without using a mathematical model, an optimal controller can be learned from data evaluated by certain performance criteria through…
In this manuscript we consider a class optimal control problem for stochastic differential delay equations. First, we rewrite the problem in a suitable infinite-dimensional Hilbert space. Then, using the dynamic programming approach, we…
In optimal control problems of control-affine systems, whose solutions are bang-bang or singular type, verification of optimality using the Hamilton-Jacobi-Bellman (HJB) equation involves the computation of partial derivatives of switching…