Related papers: On the stability of Lipschitz continuous control p…
For an infinite-horizon control problem, the optimal control can be represented by the stable manifold of the characteristic Hamiltonian system of Hamilton-Jacobi-Bellman (HJB) equation in a semiglobal domain. In this paper, we first…
This paper studies the time-inconsistent MV optimal stopping problem via a game-theoretic approach to find equilibrium strategies. To overcome the mathematical intractability of direct equilibrium analysis, we propose a vanishing…
Convex Q-learning is a recent approach to reinforcement learning, motivated by the possibility of a firmer theory for convergence, and the possibility of making use of greater a priori knowledge regarding policy or value function structure.…
This paper is concerned with stochastic impulse control problems in which the running cost changes depending on the impulse control. Because of such a dependence, it brings several difficulties when the usual dynamic programming principle…
Learning optimal feedback control laws capable of executing optimal trajectories is essential for many robotic applications. Such policies can be learned using reinforcement learning or planned using optimal control. While reinforcement…
This paper introduces the Hamilton-Jacobi-Bellman Proximal Policy Optimization (HJBPPO) algorithm into reinforcement learning. The Hamilton-Jacobi-Bellman (HJB) equation is used in control theory to evaluate the optimality of the value…
This work investigates the optimal control problem for reflected McKean-Vlasov SDEs and the viscosity solutions to Hamilton-Jacobi-Bellman(HJB) equations on the Wasserstein space in terms of intrinsic derivative. It follows from the flow…
In this article, the notion of viscosity solution is introduced for the path-dependent Hamilton-Jacobi-Bellman (PHJB) equations associated with the optimal control problems for path-dependent stochastic differential equations. We identify…
In this work we investigate regularity properties of a large class of Hamilton-Jacobi-Bellman (HJB) equations with or without obstacles, which can be stochastically interpreted in form of a stochastic control system which nonlinear cost…
Commonly in reinforcement learning (RL), rewards are discounted over time using an exponential function to model time preference, thereby bounding the expected long-term reward. In contrast, in economics and psychology, it has been shown…
We unify Hamilton-Jacobi (HJ) reachability and Reinforcement Learning (RL) through a proposed running cost formulation. We prove that the resultant travel-cost value function is the unique bounded viscosity solution of a time-dependent…
Continuous-time stochastic processes underlie many natural and engineered systems. In healthcare, autonomous driving, and industrial control, direct interaction with the environment is often unsafe or impractical, motivating offline…
Despite impressive results, reinforcement learning (RL) suffers from slow convergence and requires a large variety of tuning strategies. In this paper, we investigate the ability of RL algorithms on simple continuous control tasks. We show…
We propose and analyze a randomization scheme for a general class of impulse control problems. The solution to this randomized problem is characterized as the fixed point of a compound operator which consists of a regularized nonlocal…
This work proposes an optimal safe controller minimizing an infinite horizon cost functional subject to control barrier functions (CBFs) safety conditions. The constrained optimal control problem is reformulated as a minimization problem of…
This paper studies the stochastic optimal control of jump-diffusion processes and the associated fully nonlinear backward stochastic Hamilton--Jacobi--Bellman (BSHJB) equations. We establish the dynamic programming principle (DPP) via…
Feedback controllers for port-Hamiltonian systems reveal an intrinsic inverse optimality property since each passivating state feedback controller is optimal with respect to some specific performance index. Due to the nonlinear…
In this paper we investigate a path dependent optimal control problem on the process space with both drift and volatility controls, with possibly degenerate volatility. The dynamic value function is characterized by a fully nonlinear second…
This work proposes a novel numerical scheme for solving the high-dimensional Hamilton-Jacobi-Bellman equation with a functional hierarchical tensor ansatz. We consider the setting of stochastic control, whereby one applies control to a…
Verification theorems are key results to successfully employ the dynamic programming approach to optimal control problems. In this paper we introduce a new method to prove verification theorems for infinite dimensional stochastic optimal…