Related papers: Split optimal policy iteration for LQR problems
The optimal controller design problem for a linear, first-order spatially-invariant distributed parameter system is considered. Through a case study of the Linear Quadratic Regulator (LQR) problem for the diffusion equation over the torus,…
This paper presents an algorithm to solve the infinite horizon constrained linear quadratic regulator (CLQR) problem using operator splitting methods. First, the CLQR problem is reformulated as a (finite-time) model predictive control (MPC)…
Policy iteration is one of the classical frameworks of reinforcement learning, which requires a known initial stabilizing control. However, finding the initial stabilizing control depends on the known system model. To relax this requirement…
The continuous and discrete time Linear Quadratic Regulator (LQR) theory has been used in this paper for the design of optimal analog and discrete PID controllers respectively. The PID controller gains are formulated as the optimal…
This paper deals with the long time behavior of the optimal solution of stochastic backward linear-quadratic optimal control problem over the finite time horizon. Both weak and strong turnpike properties are established under appropriate…
We present a temporal decomposition scheme for solving long-horizon optimal control problems. In the proposed scheme, the time domain is decomposed into a set of subdomains with partially overlapping regions. Subproblems associated with the…
In this paper, a class of optimization problems with nonlinear inequality constraints is discussed. Based on the ideas of sequential quadratic programming algorithm and the method of strongly sub-feasible directions, a new superlinearly…
Despite its popularity in the reinforcement learning community, a provably convergent policy gradient method for continuous space-time control problems with nonlinear state dynamics has been elusive. This paper proposes proximal gradient…
We consider multistage stochastic optimization problems involving multiple units. Each unit is a (small) control system. Static constraints couple units at each stage. We present a mix of spatial and temporal decompositions to tackle such…
We consider control-constrained linear-quadratic optimal control problems on evolving surfaces. In order to formulate well-posed problems, we prove existence and uniqueness of weak solutions for the state equation, in the sense of…
Linear dynamical systems that obey stochastic differential equations are canonical models. While optimal control of known systems has a rich literature, the problem is technically hard under model uncertainty and there are hardly any…
We consider a scenario where a system experiences a disruption, and the states (representing health values) of its components continue to reduce over time, unless they are acted upon by a controller. Given this dynamical setting, we…
This note proposes a data-driven output-feedback stabilizing policy iteration for unknown linear discrete-time systems with unmeasurable states. Existing policy iteration methods for optimal control must start from a stabilizing control…
Numerically computing global policies to optimal control problems for complex dynamical systems is mostly intractable. In consequence, a number of approximation methods have been developed. However, none of the current methods can quantify…
Nonlinear control systems with partial information to the decision maker are prevalent in a variety of applications. As a step toward studying such nonlinear systems, this work explores reinforcement learning methods for finding the optimal…
In this paper, we address Linear Quadratic Regulator (LQR) problems through a novel iterative algorithm named EXtremum-seeking Policy iteration LQR (EXP-LQR). The peculiarity of EXP-LQR is that it only needs access to a truncated…
This paper investigates the performance of Newton's method, iterative Linear Quadratic Regulator (iLQR), and Differential Dynamic Programming (DDP) in solving discrete-time optimal control problems. We offer a unified perspective on these…
We study the convergence rates of policy iteration (PI) for nonconvex viscous Hamilton--Jacobi equations using a discrete space-time scheme, where both space and time variables are discretized. We analyze the case with an uncontrolled…
In this paper we study simulation based optimization algorithms for solving discrete time optimal stopping problems. This type of algorithms became popular among practioneers working in the area of quantitative finance. Using large…
This paper studies a discrete-time stochastic control problem with linear quadratic criteria over an infinite-time horizon. We focus on a class of control systems whose system matrices are associated with random parameters involving unknown…