Related papers: Mixing time estimation in reversible Markov chains…

Let {X_n,n\geq0} be a Markov chain on a general state space X with transition probability P and stationary probability \pi. Suppose an additive component S_n takes values in the real line R and is adjoined to the chain such that…

This paper is concerned with the development of rigorous approximations to various expectations associated with Markov chains and processes having non-stationary transition probabilities. Such non-stationary models arise naturally in…

In this paper we investigate the existence of $L^{2}(\pi)$-spectral gaps for $\pi$-irreducible, positive recurrent Markov chains on general state space. We obtain necessary and sufficient conditions for the existence of…

The second largest eigenvalue of a transition matrix $P$ has connections with many properties of the underlying Markov chain, and especially its convergence rate towards the stationary distribution. In this paper, we give an asymptotic…

We prove a version of McDiarmid's bounded differences inequality for Markov chains, with constants proportional to the mixing time of the chain. We also show variance bounds and Bernstein-type inequalities for empirical averages of Markov…

We introduce the Space-Time Markov Chain Approximation (STMCA) for a general diffusion process on a finite metric graph $\Gamma$. The STMCA is a doubly asymmetric (in both time and space) random walk defined on a subdivisions of $\Gamma$,…

We consider stochastic optimization problems where data is drawn from a Markov chain. Existing methods for this setting crucially rely on knowing the mixing time of the chain, which in real-world applications is usually unknown. We propose…

We study a large class of reversible Markov chains with discrete state space and transition matrix $P_N$. We define the notion of a set of {\it metastable points} as a subset of the state space $\G_N$ such that (i) this set is reached from…

We consider the problem of finding the transition rates of a continuous-time homogeneous Markov chain under the empirical condition that the state changes at most once during a time interval of unit length. It is proven that this…

Let $(X_t)$ be a discrete time Markov chain on a general state space. It is well-known that if $(X_t)$ is aperiodic and satisfies a drift and minorization condition, then it converges to its stationary distribution $\pi$ at an exponential…

We prove that for a sequence of finite vertex-transitive graphs of increasing sizes, the cover times are asymptotically concentrated if and only if the product of the spectral-gap and the expected cover time diverges. In fact, we prove this…

Estimating the transition dynamics of controlled Markov chains is crucial in fields such as time series analysis, reinforcement learning, and system exploration. Traditional non-parametric density estimation methods often assume independent…

Through a Metropolis-like algorithm with single step computational cost of order one, we build a Markov chain that relaxes to the canonical Fermi statistics for k non-interacting particles among m energy levels. Uniformly over the…

In this paper we extend the results of the research started by the first author, in which Karlin-McGregor diagonalization of certain reversible Markov chains over countably infinite general state spaces by orthogonal polynomials was used to…

Reversibility is a key concept in Markov models and Master-equation models of molecular kinetics. The analysis and interpretation of the transition matrix encoding the kinetic properties of the model relies heavily on the reversibility…

Hybrid Gibbs samplers represent a prominent class of approximated Gibbs algorithms that utilize Markov chains to approximate conditional distributions, with the Metropolis-within-Gibbs algorithm standing out as a well-known example. Despite…

Consider a sequence of continuous-time irreducible reversible Markov chains and a sequence of initial distributions, $\mu_n$. The sequence is said to exhibit $\mu_n$-cutoff if the convergence to stationarity in total variation distance is…

We consider the problem of estimating the asymptotic variance of a function defined on a Markov chain, an important step for statistical inference of the stationary mean. We design a novel recursive estimator that requires $O(1)$…

Exponential random graphs are used extensively in the sociology literature. This model seeks to incorporate in random graphs the notion of reciprocity, that is, the larger than expected number of triangles and other small subgraphs.…

An important problem in the implementation of Markov Chain Monte Carlo algorithms is to determine the convergence time, or the number of iterations before the chain is close to stationarity. For many Markov chains used in practice this time…