Related papers: Gradient Flows for Regularized Stochastic Control …
We explicitly construct parameter transformations between gradient flows in metric spaces, called curves of maximal slope, having different exponents when the associated function satisfies a suitable convexity condition. These…
Action-constrained reinforcement learning (ACRL) is a popular approach for solving safety-critical and resource-allocation related decision making problems. A major challenge in ACRL is to ensure agent taking a valid action satisfying…
We propose a novel continuous-time algorithm for inequality-constrained convex optimization inspired by proportional-integral control. Unlike the popular primal-dual gradient dynamics, our method includes a proportional term to control the…
Sampling a target probability distribution with an unknown normalization constant is a fundamental challenge in computational science and engineering. Recent work shows that algorithms derived by considering gradient flows in the space of…
We establish a variety of results extending the well-known Pontryagin maximum principle of optimal control to discrete-time optimal control problems posed on smooth manifolds. These results are organized around a new theorem on critical and…
We provide a new understanding of the stochastic gradient bandit algorithm by showing that it converges to a globally optimal policy almost surely using \emph{any} constant learning rate. This result demonstrates that the stochastic…
This work establishes a general stochastic maximum principle for partially observed optimal control of semi-linear stochastic partial differential equations in a nonconvex control domain. The state evolves in a Hilbert space driven by a…
This work establishes two versions of the Pontryagin-type maximum principles for partially observed optimal control of coupled forward stochastic partial differential equations (FSPDEs) and backward stochastic differential equations (BSDEs)…
In this work, we present an efficient gradient projection method for solving a class of stochastic optimal control problem with expected integral state constraint. The first order optimality condition system consisting of forward-backward…
We study the convergence problem of mean-field control theory in the presence of state constraints and non-degenerate idiosyncratic noise. Our main result is the convergence of the value functions associated to stochastic control problems…
Consider convex optimization problems subject to a large number of constraints. We focus on stochastic problems in which the objective takes the form of expected values and the feasible set is the intersection of a large number of convex…
This paper focuses on finding approximate solutions to stochastic optimal control problems with control domains being not necessarily convex, where the state trajectory is subject to controlled stochastic differential equations. The…
We study an explicit mirror-descent method for finite-horizon deterministic optimal control problems. The method is motivated by Pontryagin's maximum principle: at each iteration, one solves the state and adjoint equations and updates the…
Entropy regularization is an efficient technique for encouraging exploration and preventing a premature convergence of (vanilla) policy gradient methods in reinforcement learning (RL). However, the theoretical understanding of…
We establish a Pontryagin maximum principle for discrete time optimal control problems under the following three types of constraints: a) constraints on the states pointwise in time, b) constraints on the control actions pointwise in time,…
This paper considers the problem of partially observed optimal control for forward stochastic systems which are driven by Brownian motions and an independent Poisson random measure with a feature that the cost functional is of mean-field…
In this contribution, we present a full overview of the continuous stochastic gradient (CSG) method, including convergence results, step size rules and algorithmic insights. We consider optimization problems in which the objective function…
In this paper, we derive a version of the Pontryagin maximum principle for general finite-dimensional nonlinear optimal sampled-data control problems. Our framework is actually much more general, and we treat optimal control problems for…
This paper presents a novel stochastic gradient descent algorithm for constrained optimization. The proposed algorithm randomly samples constraints and components of the finite sum objective function and relies on a relaxed logarithmic…
This paper outlines a novel extension of the classical Pontryagin minimum (maximum) principle to stochastic optimal control problems. Contrary to the well-known stochastic Pontryagin minimum principle involving forward-backward stochastic…