Related papers: Fixed Points of the Set-Based Bellman Operator

The problem of determining the European-style option price in the incomplete market has been examined within the framework of stochastic optimization. An analytic method based on the discrete dynamic programming equation (Bellman equation)…

Dynamic programming (DP) is a fundamental tool used across many engineering fields. The main goal of DP is to solve Bellman's optimality equations for a given Markov decision process (MDP). Standard methods like policy iteration exploit the…

This paper studies the mean-field Markov decision process (MDP) with the centralized stopping under the non-exponential discount. The problem differs fundamentally from most existing studies on mean-field optimal control/stopping due to its…

In robust Markov decision processes (MDPs), the uncertainty in the transition kernel is addressed by finding a policy that optimizes the worst-case performance over an uncertainty set of MDPs. While much of the literature has focused on…

We consider the problem of quantifying uncertainty over expected cumulative rewards in model-based reinforcement learning. In particular, we focus on characterizing the variance over values induced by a distribution over MDPs. Previous work…

We study the convergence of random function iterations for finding an invariant measure of the corresponding Markov operator. We call the problem of finding such an invariant measure the stochastic fixed point problem. This generalizes…

In this present article, we get sufficient conditions for the existence and uniqueness of fixed points and common fixed points for single and double mapping satisfying various contractive conditions within the partially ordered…

We establish a coupled fixed points theorem for a meaningful class of mixed monotone multivalued operators and then we use it to derive some results on existence of quasisolutions and solutions to first--order functional differential…

We develop an exhaustive study of Markov decision process (MDP) under mean field interaction both on states and actions in the presence of common noise, and when optimization is performed over open-loop controls on infinite horizon. Such…

In this paper we study the existence and uniqueness of fixed points of a class of mappings defined on complete, (sequentially compact) cone metric spaces, without continuity conditions and depending on another function.

The Bellman operator constitutes the foundation of dynamic programming (DP). An alternative is presented by the Gauss-Seidel operator, whose evaluation, differently from that of the Bellman operator where the states are all processed at…

We revisit the well-studied superhedging problem under proportional transaction costs in continuous time using the recently developed tools of set-valued stochastic analysis. By relying on a simple Black-Scholes-type market model for…

New approaches to the theory of dynamic programming view dynamic programs as families of policy operators acting on partially ordered sets. In this paper, we extend these ideas by shifting from arbitrary partially ordered sets to ordered…

Any maximal monotone operator can be characterized by a convex function. The family of such convex functions is invariant under a transformation connected with the Fenchel-Legendre conjugation. We prove that there exist a convex…

Motivated by many application problems, we consider Markov decision processes (MDPs) with a general loss function and unknown parameters. To mitigate the epistemic uncertainty associated with unknown parameters, we take a Bayesian approach…

This paper investigates discrete-time Markov decision processes with recursive utilities (or payoffs) defined by the classic CES aggregator and the Kreps-Porteus certainty equivalent operator. According to the classification introduced by…

In this article, we derive a common fixed point result for a pair of single valued and set-valued mappings on a metric space having graphical structure. In this case, the set-valued map is assumed to be closed valued instead of closed and…

Markov decision problems are most commonly solved via dynamic programming. Another approach is Bellman residual minimization, which directly minimizes the squared Bellman residual objective function. However, compared to dynamic…

Policy iteration and value iteration are at the core of many (approximate) dynamic programming methods. For Markov Decision Processes with finite state and action spaces, we show that they are instances of semismooth Newton-type methods to…

The design of fixed point algorithms is at the heart of monotone operator theory, convex analysis, and of many modern optimization problems arising in machine learning and control. This tutorial reviews recent advances in understanding the…