Related papers: Value-Gradient based Formulation of Optimal Contro…
We study the problem of learning the optimal control policy for fine-tuning a given diffusion process, using general value function approximation. We develop a new class of algorithms by solving a variational inequality problem based on the…
Hamilton-Jacobi partial differential equations (HJ PDEs) have deep connections with a wide range of fields, including optimal control, differential games, and imaging sciences. By considering the time variable to be a higher dimensional…
Tuning hyperparameters of learning algorithms is hard because gradients are usually unavailable. We compute exact gradients of cross-validation performance with respect to all hyperparameters by chaining derivatives backwards through the…
We consider a stochastic optimal control problem governed by a stochastic differential equation with delay in the control. Using a result of existence and uniqueness of a sufficiently regular mild solution of the associated…
We deal with the convergence of the value function of an approximate control problem with uncertain dynamics to the value function of a nonlinear optimal control problem. The assumptions on the dynamics and the costs are rather general and…
The design of the performance index, also referred to as cost or reward shaping, is central to both optimal control and reinforcement learning, as it directly determines the behaviors, trade-offs, and objectives that the resulting control…
In this paper we present a novel sampling-based numerical scheme designed to solve a certain class of stochastic optimal control problems, utilizing forward and backward stochastic differential equations (FBSDEs). By means of a nonlinear…
Presented is a method for efficient computation of the Hamilton-Jacobi (HJ) equation for time-optimal control problems using the generalized Hopf formula. Typically, numerical methods to solve the HJ equation rely on a discrete grid of the…
Selecting the best hyperparameters for a particular optimization instance, such as the learning rate and momentum, is an important but nonconvex problem. As a result, iterative optimization methods such as hypergradient descent lack global…
In this paper we propose a new way of proving the value of a firm that is currently producing a certain product and faces the option to exit the market. The problem of optimal exiting is an optimal stopping problem, that can be solved using…
We study a new two-time-scale stochastic gradient method for solving optimization problems, where the gradients are computed with the aid of an auxiliary variable under samples generated by time-varying MDPs controlled by the underlying…
The purpose of this paper is to describe the numerical solution of the Hamilton-Jacobi-Bellman (HJB) for an optimal control problem for quantum spin systems. This HJB equation is a first order nonlinear partial differential equation defined…
A new approach to solving two-point boundary value problems for a wave equation is developed. This new approach exploits the principle of stationary action to reformulate and solve such problems in the framework of optimal control. In…
We address finding the semi-global solutions to optimal feedback control and the Hamilton--Jacobi--Bellman (HJB) equation. Using the solution of an HJB equation, a feedback optimal control law can be implemented in real-time with minimum…
A piecewise constant Mayer cost function is used to model optimal control problems in which the state space is partitioned into several regions, each having its own Mayer cost value. In such a context, the standard numerical methods used in…
In this paper we make a survey on the so called randomization method, a recent methodology to study stochastic optimization problems. It allows to represent the value function of an optimal control problem by a suitable backward stochastic…
Many optimal control problems are formulated as two point boundary value problems (TPBVPs) with conditions of optimality derived from the Hamilton-Jacobi-Bellman (HJB) equations. In most cases, it is challenging to solve HJBs due to the…
We consider an optimal control on networks in the spirit of the works of Achdou et al. (2013) and Imbert et al. (2013). The main new feature is that there are entry (or exit) costs at the edges of the network leading to a possible…
We study the optimal control of path-dependent piecewise deterministic processes. An appropriate dynamic programming principle is established. We prove that the associated value function is the unique minimax solution of the corresponding…
A general time-inconsistent optimal control problem is considered for stochastic differential equations with deterministic coefficients. Under suitable conditions, a Hamilton-Jacobi-Bellman type equation is derived for the equilibrium value…