Related papers: Moderate deviation principle for exponentially erg…

The Birkhoff Ergodic Theorem establishes pointwise convergence for integrable observables, but for $f\notin L^1$, no normalization yields almost sure convergence. This paper investigates trimmed ergodic sums, where the largest observations…

We give computable bounds on the rate of convergence of the transition probabilities to the stationary distribution for a certain class of geometrically ergodic Markov chains. Our results are different from earlier estimates of Meyn and…

e study the properties of the mean-type mappings ${\bf M}\colon I^p \to I^p$ of the form $${\bf M}(x_1,\dots,x_p):=\big(M_1(x_{\alpha_{1,1}},\dots,x_{\alpha_{1,d_1}}),\dots,M_p(x_{\alpha_{p,1}},\dots,x_{\alpha_{p,d_p}})\big),$$ where $p$…

Random sampling of large Markov matrices with a tunable spectral gap, a nonuniform stationary distribution, and a nondegenerate limiting empirical spectral distribution (ESD) is useful. Fix $c>0$ and $p>0$. Let $A_n$ be the adjacency matrix…

We consider the problem of approximating the stationary distribution of an ergodic Markov chain given a set of sampled transitions. Classical simulation-based approaches assume access to the underlying process so that trajectories of…

Mahlmann and Schindelhauer (2005) defined a Markov chain which they called $k$-Flipper, and showed that it is irreducible on the set of all connected regular graphs of a given degree (at least 3). We study the 1-Flipper chain, which we call…

Robust Markov Decision Processes (MDPs) are receiving much attention in learning a robust policy which is less sensitive to environment changes. There are an increasing number of works analyzing sample-efficiency of robust MDPs. However,…

Time-homogeneous Markov chains are often used as disease progression models in studies of cost-effectiveness and optimal decision-making. Maximum likelihood estimation of these models can be challenging when data are collected at a time…

Markov chain Monte Carlo is widely used in a variety of scientific applications to generate approximate samples from intractable distributions. A thorough understanding of the convergence and mixing properties of these Markov chains can be…

We formulate some simple conditions under which a Markov chain may be approximated by the solution to a differential equation, with quantifiable error probabilities. The role of a choice of coordinate functions for the Markov chain is…

Calculating averages with respect to multimodal probability distributions is often necessary in applications. Markov chain Monte Carlo (MCMC) methods to this end, which are based on time averages along a realization of a Markov process…

Moderate deviation principles (MDPs) for random walks on covering graphs with groups of polynomial volume growth are discussed in a geometric point of view. They deal with any intermediate spatial scalings between those of laws of large…

We introduce a class of Markov chains, that contains the model of stochastic approximation by averaging and non-averaging. Using martingale approximation method, we establish various deviation inequalities for separately Lipschitz functions…

We establish a moderate deviations principle (MDP) for the log-determinant $\log | \det (M_n) |$ of a Wigner matrix $M_n$ matching four moments with either the GUE or GOE ensemble. Further we establish Cram\'er--type moderate deviations and…

Consider the state space model (X_t,Y_t), where (X_t) is a Markov chain, and (Y_t) are the observations. In order to solve the so-called filtering problem, one has to compute L(X_t|Y_1,...,Y_t), the law of X_t given the observations…

Using the renewal approach we prove exponential inequalities for additive functionals and empirical processes of ergodic Markov chains, thus obtaining counterparts of inequalities for sums of independent random variables. The inequalities…

In this paper we investigate the continuum limits of a class of Markov chains. The investigation of such limits is motivated by the desire to model very large networks. We show that under some conditions, a sequence of Markov chains…

Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…

We introduce a unified operator-theoretic framework for analyzing mixing times of finite-state ergodic Markov chains that applies to both reversible and non-reversible dynamics. The central object in our analysis is the projected transition…

We characterize the invariant filtering measures resulting from Kalman filtering with intermittent observations (\cite{Bruno}), where the observation arrival is modeled as a Bernoulli process. In \cite{Riccati-weakconv}, it was shown that…