Related papers: Perov's Contraction Principle and Dynamic Programm…
This paper shows the usefulness of the Perov contraction theorem, which is a generalization of the classical Banach contraction theorem, for solving Markov dynamic programming problems. When the reward function is unbounded, combining an…
In many branches of engineering, Banach contraction mapping theorem is employed to establish the convergence of certain deterministic algorithms. Randomized versions of these algorithms have been developed that have proved useful in…
The principle of optimality is a fundamental aspect of dynamic programming, which states that the optimal solution to a dynamic optimization problem can be found by combining the optimal solutions to its sub-problems. While this principle…
This paper deals with unconstrained discounted continuous-time Markov decision processes in Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above…
While exploring dynamical systems, we often come across the principle of contraction mapping, or better known as the Banach fixed point theorem. It is an essential concept based on successive approximation, whose utility comes from two main…
This paper presents an axiomatic approach to finite Markov decision processes where the discount rate is zero. One of the principal difficulties in the no discounting case is that, even if attention is restricted to stationary policies, a…
Methods for proving functional limit laws are developed for sequences of stochastic processes which allow a recursive distributional decomposition either in time or space. Our approach is an extension of the so-called contraction method to…
Block-coordinate descent (BCD) is a popular framework for large-scale regularized optimization problems with block-separable structure. Existing methods have several limitations. They often assume that subproblems can be solved exactly at…
Contraction theory is an analytical tool to study differential dynamics of a non-autonomous (i.e., time-varying) nonlinear system under a contraction metric defined with a uniformly positive definite matrix, the existence of which results…
The paper introduces the first formulation of convex Q-learning for Markov decision processes with function approximation. The algorithms and theory rest on a relaxation of a dual of Manne's celebrated linear programming characterization of…
In this article, we provide a novel and broadly-applicable contraction-theoretic approach to continuous-time time-varying convex optimization. For any parameter-dependent contracting dynamics, we show that the tracking error is…
This paper extends the core results of discrete time infinite horizon dynamic programming to the case of state-dependent discounting. We obtain a condition on the discount factor process under which all of the standard optimality results…
In this paper, we study a Markov decision process with a non-linear discount function and with a Borel state space. We define a recursive discounted utility, which resembles non-additive utility functions considered in a number of models in…
This paper studies the dynamic programming principle using the measurable selection method for stochastic control of continuous processes. The novelty of this work is to incorporate intermediate expectation constraints on the canonical…
We study existence and uniqueness of the fixed points solutions of a large class of non-linear variable discounted transfer operators associated to a sequential decision-making process. We establish regularity properties of these solutions,…
This paper studies the dynamic programming principle for general convex stochastic optimization problems introduced by Rockafellar and Wets in [30]. We extend the applicability of the theory by relaxing compactness and boundedness…
We introduce the Blackwell discount factor for Markov Decision Processes (MDPs). Classical objectives for MDPs include discounted, average, and Blackwell optimality. Many existing approaches to computing average-optimal policies solve for…
In this paper, we extend the Banach contraction principle to metric-like as well as partial metric spaces (not essentially complete) equipped with an arbitrary binary relation. Thereafter, we derive some fixed point results which are…
We introduce the concept of a control contraction metric, extending contraction analysis to constructive nonlinear control design. We derive sufficient conditions for exponential stabilizability of all trajectories of a nonlinear control…
In order to bring contraction analysis into the very fruitful and topical fields of stochastic and Bayesian systems, we extend here the theory describes in \cite{Lohmiller98} to random differential equations. We propose new definitions of…