Related papers: Stochastic dynamic programming under recursive Eps…

We develop value iteration-based algorithms to solve in a unified manner different classes of combinatorial zero-sum games with mean-payoff type rewards. These algorithms rely on an oracle, evaluating the dynamic programming operator up to…

We present a general black box theorem that ensures convergence of a sequence of stationary Markov processes, provided a few assumptions are satisfied. This theorem relies on a control of the resolvents of the sequence of Markov processes,…

We develop a new framework for deriving time-uniform concentration bounds for the output of stochastic sequential algorithms satisfying certain recursive inequalities akin to those defining the almost-supermartingale processes introduced by…

In this paper, a convex optimization-based method is proposed for numerically solving dynamic programs in continuous state and action spaces. The key idea is to approximate the output of the Bellman operator at a particular state by the…

We are interested in the analysis of very large continuous-time Markov chains (CTMCs) with many distinct rates. Such models arise naturally in the context of reliability analysis, e.g., of computer network performability analysis, of power…

We show how to efficiently solve problems involving a quantitative measure, here called energy, as well as a qualitative acceptance condition, expressed as a B\"uchi or Parity objective, in finite weighted automata and in one-clock weighted…

The centralized training for decentralized execution paradigm emerged as the state-of-the-art approach to $\epsilon$-optimally solving decentralized partially observable Markov decision processes. However, scalability remains a significant…

We study dynamic mechanism design in a pure-exchange economy with privately observed idiosyncratic income. In the standard infinitely lived hidden-income benchmark of Green (1987) and Thomas-Worrall (1990), constrained-efficient allocations…

We study backward stochastic differential equations (BSDEs) in infinite horizon and design efficient numerical schemes for solving them. We establish a probabilistic representation of the solution of the BSDE using Malliavin derivative and…

First-order methods are often analyzed via their continuous-time models, where their worst-case convergence properties are usually approached via Lyapunov functions. In this work, we provide a systematic and principled approach to find and…

The aim of this survey is twofold. First we show how the Markov tower construction is applicable for obtaining finer stochastic properties, like a local limit theorem of probability theory. Here the fundamental method is the study of the…

We derive a system of fixed-point equations for the equilibrium transfers in a class of one-to-one matching models with linear transferable utility. We then show that, when the degree of substitution between alternatives is bounded from…

In this article, we primarily propose a novel Bayesian characterization of stationary and nonstationary stochastic processes. In practice, this theory aims to distinguish between global stationarity and nonstationarity for both parametric…

Under the assumption of no-arbitrage, the pricing of American and Bermudan options can be casted into optimal stopping problems. We propose a new adaptive simulation based algorithm for the numerical solution of optimal stopping problems in…

The stochastic convective Brinkman-Forchheimer (SCBF) equations in an open connected set $\mathcal{O}\subseteq\mathbb{R}^d$ ($d\in \{2,3,4\}$) or torus are considered in this work. We show the existence of a pathwise unique strong solution…

We study the online estimation of the optimal policy of a Markov decision process (MDP). We propose a class of Stochastic Primal-Dual (SPD) methods which exploit the inherent minimax duality of Bellman equations. The SPD methods update a…

This paper addresses the portfolio selection problem for nonlinear law-dependent preferences in continuous time, which inherently exhibit time inconsistency. Employing the method of stochastic maximum principle, we establish verification…

This article generalizes the work of Ballmann and \'Swiatkowski to the case of Reflexive Banach spaces and uniformly convex Busemann spaces, thus giving a new fixed point criterion for groups acting on simplicial complexes.

We address payoff-based decentralized learning in infinite-horizon zero-sum Markov games. In this setting, each player makes decisions based solely on received rewards, without observing the opponent's strategy or actions nor sharing…

We consider the pricing of European-style structured credit payoff in a static framework, where the underlying default times are independent given a common factor. A practical application would consist of the pricing of nth-to-default…