Related papers: Markov chains in a field of traps

We consider Markov decision processes (MDP) as generators of sequences of probability distributions over states. A probability distribution is p-synchronizing if the probability mass is at least p in a single state, or in a given set of…

We analyze frustrated states of the one-dimensional dilute Ising chain with charged interacting impurities of two types with mapping of the system to some Markov chain. We perform classification and reveal two types of Markov chains:…

We formulate some simple conditions under which a Markov chain may be approximated by the solution to a differential equation, with quantifiable error probabilities. The role of a choice of coordinate functions for the Markov chain is…

Approximating the stationary probability of a state in a Markov chain through Markov chain Monte Carlo techniques is, in general, inefficient. Standard random walk approaches require $\tilde{O}(\tau/\pi(v))$ operations to approximate the…

Interval Markov chains extend classical Markov chains with the possibility to describe transition probabilities using intervals, rather than exact values. While the standard formulation of interval Markov chains features closed intervals,…

We propose a new definition of metastability of Markov processes on countable state spaces. We obtain sufficient conditions for a sequence of processes to be metastable. In the reversible case these conditions are expressed in terms of the…

We investigate the mixing properties of a finite Markov chain in random environment defined as a mixture of a deterministic chain and a chain whose state space has been permuted uniformly at random. This work is the counterpart of a…

This paper presents a novel theoretical Monte Carlo Markov chain procedure in the framework of graphs. It specifically deals with the construction of a Markov chain whose empirical distribution converges to a given reference one. The Markov…

A method of constructing Markov chains on finite state spaces is provided. The chain is specified by three constraints: stationarity, dependence and marginal distributions. The generalized Pythagorean theorem in information geometry plays a…

We consider continuous-time Markov chains which display a family of wells at the same depth. We provide sufficient conditions which entail the convergence of the finite-dimensional distributions of the order parameter to the ones of a…

The hitting and mixing times are two fundamental quantities associated with Markov chains. In Peres and Sousi[PS2015] and Oliveira[Oli2012], the authors show that the mixing times and "worst-case" hitting times of reversible Markov chains…

Let $X_n, n \ge 0$ be a Markov chain with finite state space $M$. If $x,y \in M$ such that $x$ is transient we have $P^y(X_n = x) \to 0$ for $n \to \infty$, and under mild aperiodicity conditions this convergence is monotone in that for…

For $N\in\mathbb{N}$, let $\pi_N$ be the law of the number of fixed points of a random permutation of $\{1, 2, ..., N\}$. Let $\mathcal{P}$ be a Poisson law of parameter 1.A classical result shows that $\pi_N$ converges to $\mathcal{P}$ for…

We consider a Markov chain obtained by random iterations of Lipschitz maps $T_i$ chosen with a probability $p_i(x)$ depending on the current position $x$. We assume this system has a property of "contraction on average", that is $\sum_i…

For a random quantum state on $H=C^d \otimes C^d$ obtained by partial tracing a random pure state on $H \otimes C^s$, we consider the whether it is typically separable or typically entangled. For this problem, we show the existence of a…

Consider a Markov chain $\{X_n\}_{n\ge 0}$ with an ergodic probability measure $\pi$. Let $\Psi$ a function on the state space of the chain, with $\alpha$-tails with respect to $\pi$, $\alpha\in (0,2)$. We find sufficient conditions on the…

We study the problem of characterizing the expected hitting times for a robust generalization of continuous-time Markov chains. This generalization is based on the theory of imprecise probabilities, and the models with which we work…

Suppose that $\mathcal C$ is a finite collection of patterns. Observe a Markov chain until one of the patterns in $\mathcal C$ occurs as a run. This time is denoted by $\tau$. In this paper, we aim to give an easy way to calculate the mean…

This work introduces a notion of approximate probabilistic trace equivalence for labelled Markov chains, and relates this new concept to the known notion of approximate probabilistic bisimulation. In particular this work shows that the…

In the investigation of limits of Markov chains, the presence of states which become instantaneous states in the limit may prevent the convergence of the chain in the Skorohod topology. We present in this article a weaker topology adapted…