Related papers: Regularity of harmonic functions for some Markov c…
Two different Markov jump processes driven out of equilibrium by constant thermodynamic forces may have identical current fluctuations in the stationary state. The concept of dynamical equivalence classes emerges from this statement as…
We consider a family of Markov chains whose transition dynamics are affected by model parameters. Understanding the parametric dependence of (complex) performance measures of such Markov chains is often of significant interest. The…
This paper studies the exponential stability of random matrix products driven by a general (possibly unbounded) state space Markov chain. It is a cornerstone in the analysis of stochastic algorithms in machine learning (e.g. for parameter…
This paper gathers together different conditions which are all equivalent to geometric ergodicity of time-homogeneous Markov chains on general state spaces. A total of 34 different conditions are presented (27 for general chains plus 7 just…
Large deviation results are given for a class of perturbed nonhomogeneous Markov chains on finite state space which formally includes some stochastic optimization algorithms. Specifically, let {P_n} be a sequence of transition matrices on a…
We revisit the classical problem of approximating a stochastic differential equation by a discrete-time and discrete-space Markov chain. Our construction iterates Caratheodory's theorem over time to match the moments of the increments…
In this work, we characterise the statistics of Markov chains by constructing an associated sequence of periodic differential operators. Studying the density of states of these operators reveals the absolutely continuous invariant measure…
Continuous-time Markov chains on non-negative integers can be used for modeling biological systems, population dynamics, and queueing models. Qualitative behaviors of birth-and-death models, typical examples of such one-dimensional…
We study the ergodic behaviour of a discrete-time process $X$ which is a Markov chain in a stationary random environment. The laws of $X_t$ are shown to converge to a limiting law in (weighted) total variation distance as $t\to\infty$.…
The paper considers a stochastic differential equation of Duffing type with Markov coefficients. The existence of unpredictable solutions is considered. The unpredictability is a property of bounded functions characterized by unbounded…
We observe that the technique of Markov contraction can be used to establish measure concentration for a broad class of non-contracting chains. In particular, geometric ergodicity provides a simple and versatile framework. This leads to a…
A rescaled Markov chain converges uniformly in probability to the solution of an ordinary differential equation, under carefully specified assumptions. The presentation is much simpler than those in the outside literature. The result may be…
We study continuous time Markov processes on graphs. The notion of frequency is introduced, which serves well as a scaling factor between any Markov time of a continuous time Markov process and that of its jump chain. As an application, we…
Perturbation analysis of Markov chains provides bounds on the effect that a change in a Markov transition matrix has on the corresponding stationary distribution. This paper compares and analyzes bounds found in the literature for finite…
We consider a finite state discrete time process X. Without loss of generality the finite state space can be identified with the set of unit vectors {e1, e2, . . . , eN} with ei = (0, . . . , 0, 1, 0, . . . , 0)0 2 RN. For a Markov chain…
The paper deals with the convergence properties of the products of random (row-)stochastic matrices. The limiting behavior of such products is studied from a dynamical system point of view. In particular, by appropriately defining a dynamic…
This note studies monotone Markov chains, a subclass of Markov chains with extensive applications in operations research and economics. While the properties that ensure the global stability of these chains are well studied, their…
We formulate some simple conditions under which a Markov chain may be approximated by the solution to a differential equation, with quantifiable error probabilities. The role of a choice of coordinate functions for the Markov chain is…
Starting from a Markov chain with a finite alphabet, we consider the chain obtained when all but one symbol are undistinguishable for the practitioner. We study necessary and sufficient conditions for this chain to have continuous…
The partial sum of the states of a Markov chain or more generally a Markov source is asymptotically normally distributed under suitable conditions. One of these conditions is that the variance is unbounded. A simple combinatorial…