相关论文: Stochastic dynamic programming under recursive Eps…

We study mean field portfolio games under Epstein-Zin preferences, which naturally encompass the classical time-additive power utility as a special case. In a general non-Markovian framework, we establish a uniqueness result by proving a…

We study a catching-up algorithm for a class of differential inclusions driven by maximal monotone operators with continuous perturbations. Using a decomposition of the monotone operator into the closed convex hull of its single-valued part…

For an $\cF_T$-measurable payoff of a European type contingent claim, the recursive utility process/dynamic risk measure can be described by the adapted solution to a backward stochastic differential equation (BSDE). However, for an…

In this paper, we consider risk-sensitive Markov Decision Processes (MDPs) with Borel state and action spaces and unbounded cost under both finite and infinite planning horizons. Our optimality criterion is based on the recursive…

It is well known that mean-variance portfolio selection is a time-inconsistent optimal control problem in the sense that it does not satisfy Bellman's optimality principle and therefore the usual dynamic programming approach fails. We…

We consider policy evaluation in infinite-horizon discounted Markov decision problems (MDPs) with infinite spaces. We reformulate this task a compositional stochastic program with a function-valued decision variable that belongs to a…

Many non-trivial sequential decision-making problems are efficiently solved by relying on Bellman's optimality principle, i.e., exploiting the fact that sub-problems are nested recursively within the original problem. Here we show how it…

Discrete time stochastic optimal control problems and Markov decision processes (MDPs), respectively, serve as fundamental models for problems that involve sequential decision making under uncertainty and as such constitute the theoretical…

This paper analyzes finite state Markov Decision Processes (MDPs) with uncertain parameters in compact sets and re-examines results from robust MDP via set-based fixed point theory. To this end, we generalize the Bellman and policy…

We consider how to use the Bellman residual of the dynamic programming operator to compute suboptimality bounds for solutions to stochastic shortest path problems. Such bounds have been previously established only in the special case that…

The paper deals with a risk averse dynamic programming problem with infinite horizon. First, the required assumptions are formulated to have the problem well defined. Then the Bellman equation is derived, which may be also seen as a…

In this paper, we introduce a class of backward stochastic equations (BSEs) that extend classical BSDEs and include many interesting examples of generalized BSDEs as well as semimartingale backward equations. We show that a BSE can be…

Recently, \citet{SuttonMW15} introduced the emphatic temporal differences (ETD) algorithm for off-policy evaluation in Markov decision processes. In this short note, we show that the projected fixed-point equation that underlies ETD…

This paper deals with unconstrained discounted continuous-time Markov decision processes in Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above…

We propose empirical dynamic programming algorithms for Markov decision processes (MDPs). In these algorithms, the exact expectation in the Bellman operator in classical value iteration is replaced by an empirical estimate to get `empirical…

We study time-inconsistent recursive stochastic control problems, i.e., for which the Bellman principle of optimality does not hold. For this class of problems classical optimal controls may fail to exist, or to be relevant in practice, and…

We propose universal randomized function approximation-based empirical value iteration (EVI) algorithms for Markov decision processes. The `empirical' nature comes from each iteration being done empirically from samples available from…

We study existence, uniqueness and computability of solutions for a class of discrete time recursive utilities models. By combining two streams of the recent literature on recursive preferences---one that analyzes principal eigenvalues of…

We describe an approximate dynamic programming approach to compute lower bounds on the optimal value function for a discrete time, continuous space, infinite horizon setting. The approach iteratively constructs a family of lower bounding…

In this paper, we consider discrete-time infinite horizon problems of optimal control to a terminal set of states. These are the problems that are often taken as the starting point for adaptive dynamic programming. Under very general…