Related papers: Distributional Hamilton-Jacobi-Bellman Equations f…
Many real-world control problems, ranging from finance to robotics, evolve in continuous time with non-uniform, event-driven decisions. Standard discrete-time reinforcement learning (RL), based on fixed-step Bellman updates, struggles in…
We study the problem of generating control laws for systems with unknown dynamics. Our approach is to represent the controller and the value function with neural networks, and to train them using loss functions adapted from the…
Distributional reinforcement learning (RL) has proven useful in multiple benchmarks as it enables approximating the full distribution of returns and extracts rich feedback from environment samples. The commonly used quantile regression…
In this note, we demonstrate that a locally semiconvex viscosity supersolution to a possibly degenerate fully nonlinear elliptic Hamilton-Jacobi-Bellman (HJB) equation is differentiable along the directions spanned by the range of the…
We consider the numerical solution of Hamilton-Jacobi-Bellman equations arising in stochastic control theory. We introduce a class of monotone approximation schemes relying on monotone interpolation. These schemes converge under very weak…
We address two major challenges in scientific machine learning (SciML): interpretability and computational efficiency. We increase the interpretability of certain learning processes by establishing a new theoretical connection between…
The Hamilton-Jacobi-Bellman equation (HJB) associated with the time inhomogeneous singular control problem is a parabolic partial differential equation, and the existence of a classical solution is usually difficult to prove. In this paper,…
An optimal control problem is considered for a stochastic differential equation with the cost functional determined by a backward stochastic Volterra integral equation (BSVIE, for short). This kind of cost functional can cover the general…
This paper introduces a reinforcement learning-based tracking control approach for a class of nonlinear systems using neural networks. In this approach, adversarial attacks were considered both in the actuator and on the outputs. This…
Controlling systems of ordinary differential equations (ODEs) is ubiquitous in science and engineering. For finding an optimal feedback controller, the value function and associated fundamental equations such as the Bellman equation and the…
In this paper we establish periodic homogenization for Hamilton-Jacobi-Bellman (HJB) equations, associated to nonlocal operators of integro-differential type. We consider the case when the fractional diffusion has the same order as the…
We present a distributional approach to theoretical analyses of reinforcement learning algorithms for constant step-sizes. We demonstrate its effectiveness by presenting simple and unified proofs of convergence for a variety of…
This note lays part of the theoretical ground for a definition of differential systems modeling reinforcement learning in continuous time non-Markovian rough environments. Specifically we focus on optimal relaxed control of rough equations…
Solving the Hamilton-Jacobi-Bellman equation is important in many domains including control, robotics and economics. Especially for continuous control, solving this differential equation and its extension the Hamilton-Jacobi-Isaacs…
We consider a deterministic optimal control problem with a maximum running cost functional, in a finite horizon context, and propose deep neural network approximations for Bellman's dynamic programming principle, corresponding also to some…
Using the Hamilton-Jacobi method, we solve chemical Fokker-Planck equations within the Gaussian approximation and obtain a simple and compact formula for a conditional probability distribution. The formula holds in general transient…
In this paper, we present a novel algorithm named synchronous integral Q-learning, which is based on synchronous policy iteration, to solve the continuous-time infinite horizon optimal control problems of input-affine system dynamics. The…
In this paper we provide a rate of convergence for periodic homogenization of Hamilton-Jacobi-Bellman equations with nonlocal diffusion. The result is based on the regularity of the associated effective problem, where the convexity plays a…
The approximation of solutions to second order Hamilton--Jacobi--Bellman (HJB) equations by deep neural networks is investigated. It is shown that for HJB equations that arise in the context of the optimal control of certain Markov…
We propose a new numerical method for solving the Hamilton-Jacobi-Bellman quasi-variational inequality associated with the combined impulse and stochastic optimal control problem over a finite time horizon. Our method corresponds to an…