Related papers: Markov chains in a field of traps
We study a class of Markov chains that model the evolution of a quantum system subject to repeated measurements. Each Markov chain in this class is defined by a measure on the space of matrices. It is then given by a random product of…
This note studies monotone Markov chains, a subclass of Markov chains with extensive applications in operations research and economics. While the properties that ensure the global stability of these chains are well studied, their…
In order to approximate a continuous time stochastic process by discrete time Markov chains one has several options to embed the Markov chains into continuous time processes. On the one hand there is the Markov embedding, which uses…
Many problems of practical interest rely on Continuous-time Markov chains~(CTMCs) defined over combinatorial state spaces, rendering the computation of transition probabilities, and hence probabilistic inference, difficult or impossible…
This review paper provides an introduction of Markov chains and their convergence rates which is an important and interesting mathematical topic which also has important applications for very widely used Markov chain Monte Carlo (MCMC)…
We consider trap models on Z^d, namely continuous time Markov jump process on Z^d with embedded chain given by a generic discrete time random walk, and whose mean waiting time at x is given by tau_x, with tau = (tau_x, x in Z^d) a family of…
Given Markov chains and Markov decision processes (MDPs) whose transitions are labelled with non-negative integer costs, we study the computational complexity of deciding whether the probability of paths whose accumulated cost satisfies a…
We study the limiting object of a sequence of Markov chains analogous to the limits of graphs, hypergraphs, and other objects which have been studied. Following a suggestion of Aldous, we assign to a sequence of finite Markov chains with…
We show that an entanglement measure called relative entropy of entanglement satisfies a strong continuity condition. If two states are close to each other then so are their entanglements per particle pair in this measure. It follows in…
In this note, we prove a conditionally centered version of the quenched weak invariance principle under the Hannan condition, for stationary processes. In the course, we obtain a (new) construction of the fact that any stationary process…
Given an irreducible discrete-time Markov chain on a finite state space, we consider the largest expected hitting time $T(\alpha)$ of a set of stationary measure at least $\alpha$ for $\alpha\in(0,1)$. We obtain tight inequalities among the…
There have been extensive studies of a random walk among a field of immobile traps (or obstacles), where one is interested in the probability of survival as well as the law of the random walk conditioned on its survival up to time $t$. In…
A pair of Markov processes is called a Markov coupling if both processes have the same transition probabilities and the pair is also a Markov process. We say that a coupling is ``shy'' if the processes never come closer than some (random)…
We introduce a class of so called Markovian marginals, which gives a natural framework for constructing solutions to the quantum marginal problem. We consider a set of marginals that possess a certain internal quantum Markov chain…
We prove the existence of limiting distributions for a large class of Markov chains on a general state space in a random environment. We assume suitable versions of the standard drift and minorization conditions. In particular, the system…
The paper studies a probabilistic notion of causes in Markov chains that relies on the counterfactuality principle and the probability-raising property. This notion is motivated by the use of causes for monitoring purposes where the aim is…
We study Poisson's equation in the context of general state space Markov chains. For chains satisfying a contraction assumption w.r.t. a Wasserstein distance, we show that a solution exists for Lipschitz functions and investigate its…
We analyze the notions of monotonicity and complete monotonicity for Markov Chains in continuous-time, taking values in a finite partially ordered set. Similarly to what happens in discrete-time, the two notions are not equivalent. However,…
A classical problem for Markov chains is determining their stationary (or steady-state) distribution. This problem has an equally classical solution based on eigenvectors and linear equation systems. However, this approach does not scale to…
A variation of Rosenstock's trapping model in which $N$ independent random walkers are all initially placed upon a site of a one-dimensional lattice in the presence of a {\em one-sided} random distribution (with probability $c$) of…