Related papers: A central limit theorem for temporally non-homogen…
This paper is a survey of various proofs of the so called {\em fundamental theorem of Markov chains}: every ergodic Markov chain has a unique positive stationary distribution and the chain attains this distribution in the limit independent…
We prove a non-asymptotic central limit theorem for vector-valued martingale differences using Stein's method, and use Poisson's equation to extend the result to functions of Markov Chains. We then show that these results can be applied to…
A general theory is developed to study individual based models which are discrete in time. We begin by constructing a Markov chain model that converges to a one-dimensional map in the infinite population limit. Stochastic fluctuations are…
In this paper, we establish some functional central limit theorems for a large class of general supercritical superprocesses with spatially dependent branching mechanisms satisfying a second moment condition. In the particular case when the…
This paper investigates the limit behavior of Markov Decision Processes (MDPs) made of independent particles evolving in a common environment, when the number of particles goes to infinity. In the finite horizon case or with a discounted…
In this paper we consider the statistics of repeated measurements on the output of a quantum Markov chain. We establish a large deviations result analogous to Sanov's theorem for the empirical measure associated to finite sequences of…
We propose and analyze a temporal concatenation heuristic for solving large-scale finite-horizon Markov decision processes (MDP), which divides the MDP into smaller sub-problems along the time horizon and generates an overall solution by…
We prove a functional limit theorem for Markov chains that, in each step, move up or down by a possibly state dependent constant with probability $1/2$, respectively. The theorem entails that the law of every one-dimensional regular…
In this note, the time reversible case of a general theorem of Bhattacharya is shown to imply the Kipnis-Varadhan functional central limit theorem for ergodic Markov processes. To this end, a few results from semigroup theory, including the…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (minimize…
We study the Markov chain on $\mathbf{F}_p$ obtained by applying a function $f$ and adding $\pm\gamma$ with equal probability. When $f$ is a linear function, this is the well-studied Chung--Diaconis--Graham process. We consider two cases:…
In this paper we study the additive functionals of Markov chains via conditioning with respect to both past and future of the chain. We shall point out new sufficient projective conditions, which assure that the variance of partial sums of…
In this paper, we consider a continuous-time Markov process and prove a local limit theorem for the integral of a time-inhomogeneous function of the process. One application is in the study of the fast-oscillating perturbations of linear…
This paper surveys various results about Markov chains on general (non-countable) state spaces. It begins with an introduction to Markov chain Monte Carlo (MCMC) algorithms, which provide the motivation and context for the theory which…
The Central Limit Theorem (CLT) for additive functionals of Markov chains is a well known result with a long history. In this paper we present applications to two finite-memory versions of the Elephant Random Walk, solving a problem from…
We introduce an statistical mechanical formalism for the study of discrete-time stochastic processes with which we prove: (i) General properties of extremal chains, including triviality on the tail $\sigma$-algebra, short-range…
Constrained partially observable Markov decision processes (CPOMDPs) have been used to model various real-world phenomena. However, they are notoriously difficult to solve to optimality, and there exist only a few approximation methods for…
In the present paper, the maximum principle for finite horizon state constrained problems from the book by R. Vinter [\textit{Optimal Control}, Birkh\"auser, Boston, 2000; Theorem~9.3.1] is analyzed via parametric examples. The latter has…
We prove the Local Limit Theorems for bounded additive functionals of uniformly elliptic inhomogeneous Markov arrays. As an application we obtain the precise asymptotics in the large deviation regime for bounded additive functionals of…
In this paper we study the functional central limit theorem for stationary Markov chains with self-adjoint operator and general state space. We investigate the case when the variance of the partial sum is not asymptotically linear in n; and…