Related papers: Convergence of Policy Iteration for Entropy-Regula…
In this paper, we consider undiscouted infinite-horizon optimal control for deterministic systems with an uncountable state and input space. We specifically address the case when the classic value iteration does not converge. For such…
We study Sinkhorn's algorithm for solving the entropically regularized optimal transport problem. Its iterate $\pi_{t}$ is shown to satisfy $H(\pi_{t}|\pi_{*})+H(\pi_{*}|\pi_{t})=O(t^{-1})$ where $H$ denotes relative entropy and $\pi_{*}$…
A novel Policy Gradient (PG) algorithm, called $\textit{Matryoshka Policy Gradient}$ (MPG), is introduced and studied, in the context of fixed-horizon max-entropy reinforcement learning, where an agent aims at maximizing entropy bonuses…
We introduce a new numerical method to approximate the solution of a finite horizon deterministic optimal control problem. We exploit two Hamilton-Jacobi-Bellman PDE, arising by considering the dynamics in forward and backward time. This…
This paper investigates the convergence properties of the upwind difference scheme for the Hamilton--Jacobi--Bellman (HJB) equation, a central partial differential equation in optimal control theory. First, assuming the existence of a…
In this paper, we study the optimal singular controls for stochastic recursive systems, in which the control has two components: the regular control, and the singular control. Under certain assumptions, we establish the dynamic programming…
This paper presents a physics-informed machine learning approach for synthesizing optimal feedback control policy for infinite-horizon optimal control problems by solving the Hamilton-Jacobi-Bellman (HJB) partial differential equation(PDE).…
We study an optimal control problem for the stochastic wave equation driven by affine multiplicative noise, formulated as a stochastic linear-quadratic (SLQ) problem. By applying a stochastic Pontryagin's maximum principle, we characterize…
This paper is about operator-theoretic methods for solving nonlinear stochastic optimal control problems to global optimality. These methods leverage on the convex duality between optimally controlled diffusion processes and…
This note proposes a data-driven output-feedback stabilizing policy iteration for unknown linear discrete-time systems with unmeasurable states. Existing policy iteration methods for optimal control must start from a stabilizing control…
This paper studies the convergence of the mirror descent algorithm for finite horizon stochastic control problems with measure-valued control processes. The control objective involves a convex regularisation function, denoted as $h$, with…
In this paper we consider an energy storage optimization problem in finite time in a model with partial information that allows for a changing economic environment. The state process consists of the storage level controlled by the storage…
This paper mainly establishes the finite-horizon stochastic bounded real lemma, and then solves the $H_{\infty}$ control problem for discrete-time stochastic linear systems defined on the separable Hilbert spaces, thereby unifying the…
We study the global linear convergence of policy gradient (PG) methods for finite-horizon continuous-time exploratory linear-quadratic control (LQC) problems. The setting includes stochastic LQC problems with indefinite costs and allows…
Sample-based trajectory optimisers are a promising tool for the control of robotics with non-differentiable dynamics and cost functions. Contemporary approaches derive from a restricted subclass of stochastic optimal control where the…
In this work, we investigate a stochastic control framework for global optimization over both Euclidean spaces and the Wasserstein space of probability measures, where the objective function may be non-convex and/or non-differentiable. In…
In this paper, we consider the portfolio optimization problem in a financial market where the underlying stochastic volatility model is driven by n-dimensional Brownian motions. At first, we derive a Hamilton-Jacobi-Bellman equation…
We consider an infinite horizon control problem for dynamics constrained to remain on a multidimensional junction with entry costs. We derive the associated system of Hamilton-Jacobi equations (HJ), prove the comparison principle and that…
We consider a stochastic optimal control problem governed by a stochastic differential equation with delay in the control. Using a result of existence and uniqueness of a sufficiently regular mild solution of the associated…
Folklore says that Howard's Policy Improvement Algorithm converges extraordinarily fast, even for controlled diffusion settings. In a previous paper, we proved that approximations of the solution of a particular parabolic partial…