Related papers: Exponential Convergence and stability of Howards's…
While the optimization landscape of policy gradient methods has been recently investigated for partially observed linear systems in terms of both static output feedback and dynamical controllers, they only provide convergence guarantees to…
Estimation of Distribution Algorithms (EDAs) and Innovation Method are recognized methods for solving global optimization problems and for the estimation of parameters in diffusion processes, respectively. Well known is also that the…
This note proposes a data-driven output-feedback stabilizing policy iteration for unknown linear discrete-time systems with unmeasurable states. Existing policy iteration methods for optimal control must start from a stabilizing control…
Diffusion models have emerged as powerful tools for generative modeling, demonstrating exceptional capability in capturing target data distributions from large datasets. However, fine-tuning these massive models for specific downstream…
For linear-quadratic optimal control problems (OCPs) governed by elliptic and parabolic partial differential equations (PDEs), we investigate the impact of perturbations on optimal solutions. Local perturbations may occur, e.g., due to…
A widely used technique for improving policies is success conditioning, in which one collects trajectories, identifies those that achieve a desired outcome, and updates the policy to imitate the actions taken along successful trajectories.…
For unconstrained control problems, a local convergence rate is established for an $hp$-method based on collocation at the Radau quadrature points in each mesh interval of the discretization. If the continuous problem has a sufficiently…
In this paper, we analyze the convergence of several discretize-then-optimize algorithms, based on either a second-order or a fourth-order finite difference discretization, for solving elliptic PDE-constrained optimization or optimal…
Policy gradient methods are among the most effective methods in challenging reinforcement learning problems with large state and/or action spaces. However, little is known about even their most basic theoretical convergence properties,…
Distributionally robust optimal control (DROC) is gaining interest. This study presents a reformulation method for discrete DROC (DDROC) problems to design optimal control policies under a worst-case distributional uncertainty. The…
This work is motivated by the need to study the impact of data uncertainties and material imperfections on the solution to optimal control problems constrained by partial differential equations. We consider a pathwise optimal control…
Stochastic maximum principle of nonlinear controlled forward-backward systems, where the set of strict (classical) controls need not be convex and the diffusion coefficient depends explicitly on the variable control, is an open problem…
This paper considers a risk-constrained infinite-horizon optimal control problem and proposes to solve it in an iterative manner. Each iteration of the algorithm generates a trajectory from the starting point to the target equilibrium state…
Linear dynamical systems that obey stochastic differential equations are canonical models. While optimal control of known systems has a rich literature, the problem is technically hard under model uncertainty and there are hardly any…
Policy evaluation algorithms are essential to reinforcement learning due to their ability to predict the performance of a policy. However, there are two long-standing issues lying in this prediction problem that need to be tackled:…
Direct policy optimization in reinforcement learning is usually solved with policy-gradient algorithms, which optimize policy parameters via stochastic gradient ascent. This paper provides a new theoretical interpretation and justification…
In recent years, deep learning has been connected with optimal control as a way to define a notion of a continuous underlying learning problem. In this view, neural networks can be interpreted as a discretization of a parametric Ordinary…
This paper develops algorithms for high-dimensional stochastic control problems based on deep learning and dynamic programming. Unlike classical approximate dynamic programming approaches, we first approximate the optimal policy by means of…
We address the problem of finding the optimal policy of a constrained Markov decision process (CMDP) using a gradient descent-based algorithm. Previous results have shown that a primal-dual approach can achieve an $\mathcal{O}(1/\sqrt{T})$…
This paper investigates the robustness of stochastic optimal control for controlled regime switching diffusions. We consider systems driven by both continuous fluctuations and discrete regime changes, allowing for model misspecification in…