Related papers: Mixing time estimation in reversible Markov chains…
This paper considers the speed of convergence (mixing) of a finite Markov kernel $P$ with respect to the Kullback-Leibler divergence (entropy). Given a Markov kernel one defines either a discrete-time Markov chain (with the $n$-step…
Let $P$ be an irreducible and reversible transition matrix on a finite state space $V$ with invariant distribution $\pi$. We let $k$ chains start by choosing independent locations distributed according to $\pi$ and then they evolve…
We consider tilings of $\mathbb{Z}^2$ by two types of squares. We are interested in the rate of convergence to the stationarity of a natural Markov chain defined for square tilings. The rate of convergence can be represented by the mixing…
Ergodic properties and asymptotic stationarity are investigated in this paper for the pseudo-covariance matrix (PCM) of a recursive state estimator which is robust against parametric uncertainties and is based on plant output measurements…
In this paper, we consider the Markov-Chain Monte Carlo (MCMC) approach for random sampling of combinatorial objects. The running time of such an algorithm depends on the total mixing time of the underlying Markov chain and is unknown in…
Given a finite graph G, a vertex of the lamplighter graph consists of a zero-one labeling of the vertices of G, and a marked vertex of G. For transitive graphs G, we show that, up to constants, the relaxation time for simple random walk in…
We extend in two directions our previous results about the sampling and the empirical measures of immortal branching Markov processes. Direct applications to molecular biology are rigorous estimates of the mutation rates of polymerase chain…
In this paper we study the mixing time of the simple random walk on the giant component of supercritical $d$-dimensional random geometric graphs generated by the unit intensity Poisson Point Process in a $d$-dimensional cube of volume $n$.…
We build optimal exponential bounds for the probabilities of large deviations of sums \sum_{k=1}^nf(X_k) where (X_k) is a finite reversible Markov chain and f is an arbitrary bounded function. These bounds depend only on the stationary mean…
We study the convergence properties of a collapsed Gibbs sampler for Bayesian vector autoregressions with predictors, or exogenous variables. The Markov chain generated by our algorithm is shown to be geometrically ergodic regardless of…
Many finite-state reversible Markov chains can be naturally decomposed into "projection" and "restriction" chains. In this paper we provide bounds on the total variation mixing times of the original chain in terms of the mixing properties…
When sampling a multi-modal distribution $\pi(x)$, $x\in \rr^d$, a Markov chain with local proposals is often slowly mixing; while a Small-World sampler \citep{guankrone} -- a Markov chain that uses a mixture of local and long-range…
We prove new upper and lower bounds for sample complexity of finding an $\epsilon$-optimal policy of an infinite-horizon average-reward Markov decision process (MDP) given access to a generative model. When the mixing time of the…
The edge flipping is a non-reversible Markov chain on a given connected graph, which is defined by Chung and Graham in [CG12]. In the same paper, its eigenvalues and stationary distributions for some classes of graphs are identified. We…
Given a sequence of observations from a discrete-time, finite-state hidden Markov model, we would like to estimate the sampling distribution of a statistic. The bootstrap method is employed to approximate the confidence regions of a…
In Monte-Carlo methods the Markov processes used to sample a given target distribution usually satisfy detailed balance, i.e. they are time-reversible. However, relatively recent results have demonstrated that appropriate reversible and…
Markov chains can be used to generate samples whose distribution approximates a given target distribution. The quality of the samples of such Markov chains can be measured by the discrepancy between the empirical distribution of the samples…
The performance of Maximum a posteriori (MAP) estimation is studied analytically for binary symmetric multi-channel Hidden Markov processes. We reduce the estimation problem to a 1D Ising spin model and define order parameters that…
A spectral method for identifying lumping in large Markov chains is presented. Identification of meta stable states is treated as a special case. The method is based on spectral analysis of a self-adjoint matrix that is a function of the…
In this work, we are concerned with existence and uniqueness of invariant measures for path-dependent random diffusions and their time discretizations. The random diffusion here means a diffusion process living in a random environment…