Related papers: Fixed Points of the Set-Based Bellman Operator
We consider an infinite-dimensional non-linear operator related to a hard core (HC) model with a countable set $\mathbb{N}$ of spin values. It is known that finding the fixed points of an infinite-dimensional operator is generally…
This paper studies function approximation for finite horizon discrete time Markov decision processes under certain convexity assumptions. Uniform convergence of these approximations on compact sets is proved under several sampling schemes…
We discuss some results concerning fixed point equations in the setting of topological *-algebras of unbounded operators. In particular, an existence result is obtained for what we have called {\em weak $\tau$ strict contractions}, and some…
In this paper, we propose a general theory of ambiguity-averse MDPs, which treats the uncertain transition probabilities as random variables and evaluates a policy via a risk measure applied to its random return. This ambiguity-averse MDP…
A Markov decision process can be parameterized by a transition kernel and a reward function. Both play essential roles in the study of reinforcement learning as evidenced by their presence in the Bellman equations. In our inquiry of various…
In this paper, we propose a new general and stable fixed-point approach to compute the resolvents of the composition of a set-valued maximal monotone operator with a linear bounded mapping. Weak, strong and linear convergence of the…
We study value-iteration (VI) algorithms for solving general (a.k.a. multichain) Markov decision processes (MDPs) under the average-reward criterion, a fundamental but theoretically challenging setting. Beyond the difficulties inherent to…
In this paper, we investigate the existence and uniqueness of fixed points for self-mappings defined on bipolar metric spaces using a new class of contractive conditions, namely polynomial-type contractions. Our main results establish…
We consider large-scale Markov decision processes (MDPs) with parameter uncertainty, under the robust MDP paradigm. Previous studies showed that robust MDPs, based on a minimax approach to handle uncertainty, can be solved using dynamic…
We describe fixed points of an infinite dimensional non-linear operator related to a hard core (HC) model with a countable set $\mathbb{N}$ of spin values on the Cayley tree. This operator is defined by a countable set of parameters…
In this paper, we are concerned with the study of the existence of fixed points for single and multi-valued three-points contractions. Namely, we first introduce a new class of single-valued mappings defined on a metric space equipped with…
In this article, we investigate some fixed point results satisfying a new generalized $\Delta$-implicit contractive condition in ordered complete multiplicative $\mathbf{G}_\mathcal{M}-$metric space. Also, some new definitions and fixed…
We propose empirical dynamic programming algorithms for Markov decision processes (MDPs). In these algorithms, the exact expectation in the Bellman operator in classical value iteration is replaced by an empirical estimate to get `empirical…
This paper verifies a conjecture posed in a pair of papers on the fixed point sets for a class of quantum operations. Specifically, it is proved that if a quantum operation has mutually commuting operation elements that are effects forming…
We present the first finite-sample analysis of policy evaluation in robust average-reward Markov Decision Processes (MDPs). Prior work in this setting have established only asymptotic convergence guarantees, leaving open the question of…
This paper deals with unconstrained discounted continuous-time Markov decision processes in Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above…
Discrete time stochastic optimal control problems and Markov decision processes (MDPs), respectively, serve as fundamental models for problems that involve sequential decision making under uncertainty and as such constitute the theoretical…
Robust Markov decision processes (MDPs) aim to handle changing or partially known system dynamics. To solve them, one typically resorts to robust optimization methods. However, this significantly increases computational complexity and…
In this paper we give a concept of multi-dimensional-time dynamical system (MDTDS). Such dynamical system is generated by a finite family of functions $\{f_i\}$. The multi-dimensional-time space is taken as a free group. Using the subgroups…
In this paper we introduce a corrected extension of Burton's theory of large contractions in the context of triangle-perimeter contractions introduced by Petrov. Combining these two lines of research, we prove a fixed point result for large…