Related papers: Policy Iteration Achieves Regularized Equilibrium …
This paper concerns continuous dependence estimates for Hamilton-Jacobi-Bellman-Isaacs operators (briefly, HJBI). For the parabolic Cauchy problem, we establish such an estimate in the whole space $[0,+\infty)\times\Rn$. Moreover, under…
This paper deals with the long time behavior of solutions to the spatially homogeneous Boltzmann equation. The interactions considered are the so-called (non cut-off and non mollified) hard potentials. We prove an exponential in time…
This is a companion paper of [Mixed equilibrium solution of time-inconsistent stochastic LQ problem, arXiv:1802.03032], where general theory has been established to characterize the open-loop equilibrium control, feedback equilibrium…
This paper considers an infinite-horizon Markov decision process (MDP) that allows for general non-exponential discount functions, in both discrete and continuous time. Due to the inherent time inconsistency, we look for a randomized…
The Hamilton-Jacobi-Bellman equation (HJB) associated with the time inhomogeneous singular control problem is a parabolic partial differential equation, and the existence of a classical solution is usually difficult to prove. In this paper,…
We introduce a contractive abstract dynamic programming framework and related policy iteration algorithms, specifically designed for sequential zero-sum games and minimax problems with a general structure. Aside from greater generality, the…
In this paper, we focus on a class of time-inconsistent stochastic control problems, where the objective function includes the mean and several higher-order central moments of the terminal value of state. To tackle the time-inconsistency,…
Tackling large approximate dynamic programming or reinforcement learning problems requires methods that can exploit regularities, or intrinsic structure, of the problem in hand. Most current methods are geared towards exploiting the…
This paper investigates a continuous-time portfolio optimization problem with the following features: (i) a no-short selling constraint; (ii) a leverage constraint, that is, an upper limit for the sum of portfolio weights; and (iii) a…
An exponentially convergent numerical method for solving a differential equation with a right-hand fractional Riemann-Liouville time-derivative and an unbounded operator coefficient in Banach space is proposed and analysed for a…
We study the problem of computing the value function from a discretely-observed trajectory of a continuous-time diffusion process. We develop a new class of algorithms based on easily implementable numerical schemes that are compatible with…
The goal of this paper is to investigate new and simple convergence analysis of dynamic programming for linear quadratic regulator problem of discrete-time linear time-invariant systems. In particular, bounds on errors are given in terms of…
In this paper, we focus on the stochastic representation of a system of coupled Hamilton-Jacobi-Bellman-Isaacs (HJB-Isaacs (HJBI), for short) equations which is in fact a system of coupled Isaacs' type integral-partial differential…
In this paper we propose the design of an iterative observer using space as a time-like variable and prove its convergence. The iterative observer algorithm solves boundary estimation problem for a steady-state elliptic equation system…
Modified policy iteration (MPI) is a dynamic programming algorithm that combines elements of policy iteration and value iteration. The convergence of MPI has been well studied in the context of discounted and average-cost MDPs. In this…
Decision-making problems in uncertain or stochastic domains are often formulated as Markov decision processes (MDPs). Policy iteration (PI) is a popular algorithm for searching over policy-space, the size of which is exponential in the…
We consider the infinite-horizon discounted optimal control problem formalized by Markov Decision Processes. We focus on several approximate variations of the Policy Iteration algorithm: Approximate Policy Iteration, Conservative Policy…
For continuous systems modeled by dynamical equations such as ODEs and SDEs, Bellman's Principle of Optimality takes the form of the Hamilton-Jacobi-Bellman (HJB) equation, which provides the theoretical target of reinforcement learning…
In this paper, we present a scalable deep learning approach to solve opinion dynamics stochastic optimal control problems with mean field term coupling in the dynamics and cost function. Our approach relies on the probabilistic…
The semi-implicit Euler-Maruyama (EM) method is investigated to approximate a class of time-changed stochastic differential equations, whose drift coefficient can grow super-linearly and diffusion coefficient obeys the global Lipschitz…