Related papers: Value iteration for approximate dynamic programmin…

This paper studies stochastic optimization problems and associated Bellman equations in formats that allow for reduced dimensionality of the cost-to-go functions. In particular, we study stochastic control problems in the…

In a Markovian framework, we consider the problem of finding the minimal initial value of a controlled process allowing to reach a stochastic target with a given level of expected loss. This question arises typically in approximate hedging…

Stochastic time-varying optimization is an integral part of learning in which the shape of the function changes over time in a non-deterministic manner. This paper considers multiple models of stochastic time variation and analyzes the…

This paper considers an infinite-horizon Markov decision process (MDP) that allows for general non-exponential discount functions, in both discrete and continuous time. Due to the inherent time inconsistency, we look for a randomized…

We consider linear programming (LP) problems in infinite dimensional spaces that are in general computationally intractable. Under suitable assumptions, we develop an approximation bridge from the infinite-dimensional LP to tractable finite…

Primal-dual splitting involving proximity operators in order to be able to find some approximation to the minimizer for a general form of Tikhonov type functional is in the focus of this work. This approximation is produced by a pair of…

In this paper we present an algorithm for pricing barrier options in one-dimensional Markov models. The approach rests on the construction of an approximating continuous-time Markov chain that closely follows the dynamics of the given…

We consider a dynamic programming problem with arbitrary state space and bounded rewards. Is it possible to define in an unique way a limit value for the problem, where the "patience" of the decision-maker tends to infinity ? We consider,…

In this paper, we present a new set-valued Lagrange multiplier theorem for constrained convex set-valued optimization problems. We introduce the novel concept of Lagrange process. This concept is a natural extension of the classical concept…

In this work, we deal with an iteration method for approximating a fixed point of a contraction mapping using the Mann's algorithm under functional random errors. We first show its almost complete convergence to the fixed point by mean of…

In this paper, it is shown that Bermudan option pricing based on either the r\'eduite (in a one-dimensional setting: piecewise harmonic interpolation) or cubature -- is sensible from an economic vantage point: Any sequence of thus-computed…

Partially observable Markov decision processes (POMDPs) provide an elegant mathematical framework for modeling complex decision and planning problems in stochastic domains in which states of the system are observable only indirectly, via a…

We formally verify executable algorithms for solving Markov decision processes (MDPs) in the interactive theorem prover Isabelle/HOL. We build on existing formalizations of probability theory to analyze the expected total reward criterion…

It is well known that iterates of quasi-compact operators converge towards a spectral projection, whereas the explicit construction of the limiting operator is in general hard to obtain. Here, we show a simple method to explicitly construct…

We present a convex-concave reformulation of the reversible Markov chain estimation problem and outline an efficient numerical scheme for the solution of the resulting problem based on a primal-dual interior point method for monotone…

In many branches of engineering, Banach contraction mapping theorem is employed to establish the convergence of certain deterministic algorithms. Randomized versions of these algorithms have been developed that have proved useful in…

This paper shows the usefulness of the Perov contraction theorem, which is a generalization of the classical Banach contraction theorem, for solving Markov dynamic programming problems. When the reward function is unbounded, combining an…

Designing efficient learning algorithms with complexity guarantees for Markov decision processes (MDPs) with large or continuous state and action spaces remains a fundamental challenge. We address this challenge for entropy-regularized MDPs…

The problem of determining the (least) fixpoint of (higher-dimensional) functions over the non-negative reals frequently occurs when dealing with systems endowed with a quantitative semantics. We focus on the situation in which the…

One of the most widely used methods for solving average cost MDP problems is the value iteration method. This method, however, is often computationally impractical and restricted in size of solvable MDP problems. We propose acceleration…