Related papers: Solutions of Poisson's Equation for Stochastically…
This paper considers the Poisson equation for general state-space Markov chains in continuous time. The main purpose of this paper is to present specific bounds for the solutions of the Poisson equation for general state-space Markov…
This article shows how coupled Markov chains that meet exactly after a random number of iterations can be used to generate unbiased estimators of the solutions of the Poisson equation. Through this connection, we re-derive known unbiased…
This paper is concerned with the solution of the optimal stopping problem associated to the valuation of Perpetual American options driven by continuous time Markov chains. We introduce a new dynamic approach for the numerical pricing of…
Poisson's equation is fundamental to the study of Markov chains, and arises in connection with martingale representations and central limit theorems for additive functionals, perturbation theory for stationary distributions, and average…
Poisson's equation plays a fundamental role as a tool for performance evaluation and optimization of Markov chains. For continuous-time birth-death chains with possibly unbounded transition and cost rates as addressed herein, when…
The solution to Poisson's equation arise in many Markov chain and Markov jump process settings, including that of the central limit theorem, value functions for average reward Markov decision processes, and within the gradient formula for…
Stochastic approximation is a powerful class of algorithms with celebrated success. However, a large body of previous analysis focuses on stochastic approximations driven by contractive operators, which is not applicable in some important…
This paper is concerned with the development of rigorous approximations to various expectations associated with Markov chains and processes having non-stationary transition probabilities. Such non-stationary models arise naturally in…
We discuss Stein's method for approximation by the stationary distribution of a single-birth Markov chain, in conjunction with stochastic monotonicity and similar assumptions. We use bounds on the increments of the solution of Poisson's…
We consider the problem of estimating the asymptotic variance of a function defined on a Markov chain, an important step for statistical inference of the stationary mean. We design a novel recursive estimator that requires $O(1)$…
We study Poisson's equation in the context of general state space Markov chains. For chains satisfying a contraction assumption w.r.t. a Wasserstein distance, we show that a solution exists for Lipschitz functions and investigate its…
This paper presents a simple algorithm to check whether reachability probabilities in parametric Markov chains are monotonic in (some of) the parameters. The idea is to construct - only using the graph structure of the Markov chain and…
The aim of this paper is to establish the existence and uniqueness of the solution to a system of nonlinear fully coupled forward-backward doubly stochastic differential equations with Poisson jumps. Our system is Markovian in the sense…
The solution of Poisson's equation plays a key role in constructing the martingale through which sums of Markov correlated random variables can be analyzed. In this paper, we study two different representations for the solution in countable…
We study decision timing problems on finite horizon with Poissonian information arrivals. In our model, a decision maker wishes to optimally time her action in order to maximize her expected reward. The reward depends on an unobservable…
In this paper, we are devoted to developing matrix-analytic methods for solving Poisson's equation for irreducible and positive recurrent discrete-time Markov chains (DTMCs). Two special solutions, including the deviation matrix D and the…
The existence, uniqueness, and asymptotic stability of modulo periodic Poisson stable solutions of dynamic equations on a periodic time scale are investigated. The model under investigation involves a term which is constructed via a Poisson…
In this paper we study the Poisson Hypothesis, which is a device to analyze approximately the behavior of large queueing networks. We prove it in some simple limiting cases. We show in particular that the corresponding dynamical system,…
Poisson's equation has a lot of applications in various areas. Usually it is hard to derive the explicit expression of the solution of Poisson's equation for a Markov chain on an infinitely many state space. We will present a computational…
In this paper we prove the Poisson Hypothesis for the limiting behavior of the large queueing systems in some simple ("mean-field") cases. We show in particular that the corresponding dynamical systems, defined by the non-linear Markov…