Related papers: Is Bellman Equation Enough for Learning Control?
Optimal control and the associated second-order Hamilton-Jacobi-Bellman (HJB) equation are studied for unbounded stochastic evolution systems in Hilbert spaces. A new notion of viscosity solution, featured by absence of B-continuity, is…
This paper investigates the convergence properties of the upwind difference scheme for the Hamilton--Jacobi--Bellman (HJB) equation, a central partial differential equation in optimal control theory. First, assuming the existence of a…
The control of relaxation-type systems of ordinary differential equations is investigated using the Hamilton-Jacobi-Bellman equation. First, we recast the model as a singularly perturbed dynamics which we embed in a family of controlled…
For an infinite-horizon control problem, the optimal control can be represented by the stable manifold of the characteristic Hamiltonian system of Hamilton-Jacobi-Bellman (HJB) equation in a semiglobal domain. In this paper, we first…
This paper considers consumption and portfolio optimization problems with recursive preferences in both infinite and finite time regions. Specially, the financial market consists of a risk-free asset and a risky asset that follows a general…
Verification theorems are key results to successfully employ the dynamic programming approach to optimal control problems. In this paper we introduce a new method to prove verification theorems for infinite dimensional stochastic optimal…
As autonomous systems become more ubiquitous in daily life, ensuring high performance with guaranteed safety is crucial. However, safety and performance could be competing objectives, which makes their co-optimization difficult.…
We present an explicit solution to the discrete-time Bellman equation for minimax optimal control of positive systems under unconstrained disturbances. The primary contribution of our result relies on deducing a bound for the disturbance…
Despite impressive results, reinforcement learning (RL) suffers from slow convergence and requires a large variety of tuning strategies. In this paper, we investigate the ability of RL algorithms on simple continuous control tasks. We show…
An optimal control problem is considered for a stochastic differential equation containing a state-dependent regime switching, with a recursive cost functional. Due to the non-exponential discounting in the cost functional, the problem is…
A general bilinear optimal control problem subject to an infinite-dimensional state equation is considered. Polynomial approximations of the associated value function are derived around the steady state by repeated formal differentiation of…
Continuous-time reinforcement learning offers an appealing formalism for describing control problems in which the passage of time is not naturally divided into discrete increments. Here we consider the problem of predicting the distribution…
In this paper, we study a time-inconsistent stochastic optimal control problem with a recursive cost functional by a multi-person hierarchical differential game approach. An equilibrium strategy of this problem is constructed and a…
Convex Q-learning is a recent approach to reinforcement learning, motivated by the possibility of a firmer theory for convergence, and the possibility of making use of greater a priori knowledge regarding policy or value function structure.…
This paper presents a mathematical formulation to perform temporal parallelisation of continuous-time optimal control problems, which can be solved via the Hamilton--Jacobi--Bellman (HJB) equation. We divide the time interval of the control…
We study the exploratory Hamilton--Jacobi--Bellman (HJB) equation arising from the entropy-regularized exploratory control problem, which was formulated by Wang, Zariphopoulou and Zhou (J. Mach. Learn. Res., 21, 2020) in the context of…
Feedback controllers for port-Hamiltonian systems reveal an intrinsic inverse optimality property since each passivating state feedback controller is optimal with respect to some specific performance index. Due to the nonlinear…
Deterministic optimal impulse control problem with terminal state constraint is considered. Due to the appearance of the terminal state constraint, the value function might be discontinuous in general. The main contribution of this paper is…
This paper presents an inverse optimality method to solve the Hamilton-Jacobi-Bellman equation for a class of nonlinear problems for which the cost is quadratic and the dynamics are affine in the input. The method is inverse optimal because…
We study the problem of optimal portfolio selection under stochastic volatility within a continuous time reinforcement learning framework with portfolio constraints. Exploration is modeled through entropy-regularized relaxed controls, where…