Related papers: Convergence Analysis of Policy Iteration
We consider both discrete and continuous "uncertain horizon" deterministic control processes, for which the termination time is a random variable. We examine the dynamic programming equations for the value function of such processes,…
We consider deterministic infinite horizon optimal control problems with nonnegative stage costs. We draw inspiration from learning model predictive control scheme designed for continuous dynamics and iterative tasks, and propose a rollout…
We study an optimal control problem on infinite time horizon with semimartingale strategies, random coefficients and regime switching. The value function and the optimal strategy can be characterized in terms of three systems of backward…
Making optimal decisions under uncertainty is a shared problem among distinct fields. While optimal control is commonly studied in the framework of dynamic programming, it is approached with differing perspectives of the Bellman optimality…
Controlling systems of ordinary differential equations (ODEs) is ubiquitous in science and engineering. For finding an optimal feedback controller, the value function and associated fundamental equations such as the Bellman equation and the…
The mathematical framework of hybrid system is a recent and general tool to treat control systems involving control action of heterogeneous nature. In this paper, we construct and test a semi-Lagrangian numerical scheme for solving the…
The control of ensembles of dynamical systems is an intriguing and challenging problem, arising for example in quantum control. We initiate the investigation of optimal control of ensembles of discrete-time systems, focusing on minimising…
The article poses a general model for optimal control subject to information constraints, motivated in part by recent work of Sims and others on information-constrained decision-making by economic agents. In the average-cost optimal control…
This paper is on learning the Kalman gain by policy optimization method. Firstly, we reformulate the finite-horizon Kalman filter as a policy optimization problem of the dual system. Secondly, we obtain the global linear convergence of…
We consider the determination of the optimal stationary singular stochastic control of a linear diffusion for a class of average cumulative cost minimization problems arising in various financial and economic applications of stochastic…
We present a dynamic programming-based solution to a stochastic optimal control problem up to a hitting time for a discrete-time Markov control process. Firstly, we determine an optimal control policy to steer the process toward a compact…
In optimal control problem, policy iteration (PI) is a powerful reinforcement learning (RL) tool used for designing optimal controller for the linear systems. However, the need for an initial stabilizing control policy significantly limits…
We consider an optimal stochastic impulse control problem over an infinite time horizon motivated by a model of irreversible investment choices with fixed adjustment costs. By employing techniques of viscosity solutions and relying on…
This paper studies value iteration for infinite horizon contracting Markov decision processes under convexity assumptions and when the state space is uncountable. The original value iteration is replaced with a more tractable form and the…
We study stochastic optimal control problems for (possibly degenerate) McKean-Vlasov controlled diffusions and obtain discrete-time as well as finite interacting particle approximations. (i) Under mild assumptions, we first prove the…
We study the problem of data-driven, constrained control of unknown nonlinear dynamics from a single ongoing and finite-horizon trajectory. We consider a one-step optimal control problem with a smooth, black-box objective, typically a…
We consider approximate dynamic programming for the infinite-horizon stationary $\gamma$-discounted optimal control problem formalized by Markov Decision Processes. While in the exact case it is known that there always exists an optimal…
The convex analytic method has proved to be a very versatile method for the study of infinite horizon average cost optimal stochastic control problems. In this paper, we revisit the convex analytic method and make three primary…
This paper considers optimal control of dynamical systems which are represented by nonlinear stochastic differential equations. It is well-known that the optimal control policy for this problem can be obtained as a function of a value…
In this paper, a convex optimization-based method is proposed for numerically solving dynamic programs in continuous state and action spaces. The key idea is to approximate the output of the Bellman operator at a particular state by the…