Related papers: Value-Gradient based Formulation of Optimal Contro…
In multistage decision problems, it is often the case that an initial strategic decision (such as investment) is followed by many operational ones (operating the investment). Such initial strategic decision can be seen as a parameter…
In this paper, we study the optimal singular controls for stochastic recursive systems, in which the control has two components: the regular control, and the singular control. Under certain assumptions, we establish the dynamic programming…
We introduce a novel extension to robust control theory that explicitly addresses uncertainty in the value function's gradient, a form of uncertainty endemic to applications like reinforcement learning where value functions are…
We formulate and analyse an optimal control problem for the coagulation-fragmentation equation, where a scalar, time-dependent control modulates the coagulation rate by multiplying the coagulation kernel. The objective functional consists…
We propose a neural network approach to model general interaction dynamics and an adjoint based stochastic gradient descent algorithm to calibrate its parameters. The parameter calibration problem is considered as optimal control problem…
Off-policy learning refers to the problem of learning the value function of a way of behaving, or policy, while following a different policy. Gradient-based off-policy learning algorithms, such as GTD and TDC/GQ, converge even when using…
We study the Hamilton-Jacobi equation for undiscounted exit time control problems with general nonnegative Lagrangians using the dynamic programming approach. We prove theorems characterizing the value function as the unique…
We consider a Bayesian adaptive optimal stochastic control problem where a hidden static signal has a non-separable influence on the drift of a noisy observation. Being allowed to control the specific form of this dependence, we aim at…
This note lays part of the theoretical ground for a definition of differential systems modeling reinforcement learning in continuous time non-Markovian rough environments. Specifically we focus on optimal relaxed control of rough equations…
In this paper, we aim to develop the theory of optimal stochastic control for branching diffusion processes where both the movement and the reproduction of the particles depend on the control. More precisely, we study the problem of…
An effective form of the Variation Evolving Method (VEM), which originates from the continuous-time dynamics stability theory, is developed for the classic time-optimal control problem with control constraint. Within the mathematic…
This paper focuses on optimal control problem for a class of discrete-time nonlinear systems. In practical applications, computation time is a crucial consideration when solving nonlinear optimal control problems, especially under real-time…
Optimization problems are prevalent in various fields, and the gradient-based gradient descent algorithm is a widely adopted optimization method. However, in classical computing, computing the numerical gradient for a function with $d$…
In this paper, we study a time-inconsistent stochastic optimal control problem with a recursive cost functional by a multi-person hierarchical differential game approach. An equilibrium strategy of this problem is constructed and a…
We derive and implement a second-order adjoint method to compute exact gradients and Hessians for a prototypical quantum optimal control problem, that of solving for the minimal energy applied electric field that drives a molecule from a…
This work is the third part of a program initiated in arXiv:2111.13258, arXiv:2302.06571 aiming at the development of an intrinsic geometric well-posedness theory for Hamilton-Jacobi equations related to controlled gradient flow problems in…
We introduce a numerical method, based on modified hat functions, for solving a class of fractional optimal control problems. In our scheme, the control and the fractional derivative of the state function are considered as linear…
As autonomous systems become more ubiquitous in daily life, ensuring high performance with guaranteed safety is crucial. However, safety and performance could be competing objectives, which makes their co-optimization difficult.…
In this article, two methods for solving mean-field type optimal control problems are proposed and investigated. The two methods are iterative methods: at each iteration, a Hamilton-Jacobi-Bellman equation is solved, for a terminal…
In this paper, we consider continuous-time stochastic optimal control problems where the cost is evaluated through a coherent risk measure. We provide an explicit gradient descent-ascent algorithm which applies to problems subject to…