Related papers: Rapid mixing in Markov chains
Direct simulation of biomolecular dynamics in thermal equilibrium is challenging due to the metastable nature of conformation dynamics and the computational cost of molecular dynamics. Biased or enhanced sampling methods may improve the…
Metastability is a physical phenomenon ubiquitous in first order phase transitions. A fruitful mathematical way to approach this phenomenon is the study of rare transitions Markov chains. For Metropolis chains associated with Statistical…
We discuss problems posed by the quantitative study of time inhomogeneous Markov chains. The two main notions for our purpose are merging and stability. Merging (also called weak ergodicity) occurs when the chain asymptotically forgets…
In this note, we present few examples of Piecewise Deterministic Markov Processes and their long time behavior. They share two important features: they are related to concrete models (in biology, networks, chemistry,. . .) and they are…
This paper focuses on the design of time-invariant memoryless control policies for fully observed controlled Markov chains, with a finite state space. Safety constraints are imposed through a pre-selected set of forbidden states. A state is…
Many systems are partially stochastic in nature. We have derived data driven approaches for extracting stochastic state machines (Markov models) directly from observed data. This chapter provides an overview of our approach with numerous…
Many natural Markov chains fail to mix to their stationary distribution in polynomially many steps. Often, this slow mixing is inevitable since it is computationally intractable to sample from their stationary measure. Nevertheless, Markov…
Given a strongly stationary Markov chain and a finite set of stopping rules, we prove the existence of a polynomial algorithm which projects the Markov chain onto a minimal Markov chain without redundant information. Markov complexity is…
Using elementary methods, we prove that for a countable Markov chain $P$ of ergodic degree $d > 0$ the rate of convergence towards the stationary distribution is subgeometric of order $n^{-d}$, provided the initial distribution satisfies…
Markov chain Monte Carlo (MCMC) algorithms provide a very general recipe for estimating properties of complicated distributions. While their use has become commonplace and there is a large literature on MCMC theory and practice, MCMC users…
We establish a simple variance inequality for U-statistics whose underlying sequence of random variables is an ergodic Markov Chain. The constants in this inequality are explicit and depend on computable bounds on the mixing rate of the…
In this paper we continue the study of conditional Markov chains (CMCs) with finite state spaces, that we initiated in Bielecki, Jakubowski and Niew\k{e}g\l owski (2015). Here, we turn our attention to the study of Markov consistency and…
We establish general theorems quantifying the notion of recurrence --- through an estimation of the moments of passage times --- for irreducible continuous-time Markov chains on countably infinite state spaces. Sharp conditions of…
Analyses of serially-sampled data often begin with the assumption that the observations represent discrete samples from a latent continuous-time stochastic process. The continuous-time Markov chain (CTMC) is one such generative model whose…
This note provides several recent progresses in the study of long time behavior of Markov processes. The examples presented below are related to other scientific fields as PDE's, physics or biology. The involved mathematical tools as…
We study algorithms to analyze a particular class of Markov population processes that is often used in epidemiology. More specifically, Markov binomial chains are the model that arises from stochastic time-discretizations of classical…
We prove an invariance principle for non-stationary random processes and establish a rate of convergence under a new type of mixing condition. The dependence is exponentially decaying in the gap between the past and the future and is…
The paper is largely of a review nature. It considers two main methods used to study stability and obtain appropriate quantitative estimates of perturbations of (inhomogeneous) Markov chains with continuous time and a finite or countable…
We study a Markov chain with very different mixing rates depending on how mixing is measured. The chain is the "Burnside process on the hypercube $C_2^n$." Started at the all-zeros state, it mixes in a bounded number of steps, no matter how…
The mean time taken by an irreducible Markov chain on a finite state space to hit a target chosen at random according to the stationary distribution does not depend on the initial state of the chain. This mean time is known as Kemeny's…