Related papers: Walks on chains
This project is going to work with one example of stochastic matrix to understand how Markov chains evolve and how to use them to make faster and better decisions only looking to the present state of the system.
We consider two types of discrete-time Markov chains where the state space is a graded poset and the transitions are taken along the covering relations in the poset. The first type of Markov chain goes only in one direction, either up or…
This short note is devoted to motivate and clarify the notion of sequential walk introduced by the authors in a previous work. We also give some applications of this concept.
The tail chain of a Markov chain can be used to model the dependence between extreme observations. For a positive recurrent Markov chain, the tail chain aids in describing the limit of a sequence of point processes $\{N_n,n\geq1\}$,…
The reciprocal class of a Markov path measure is the set of all mixtures of its bridges. We give characterizations of the reciprocal class of a continuous-time Markov random walk on a graph. Our main result is in terms of some reciprocal…
This chapter surveys progress on three related topics in perturbations of Markov chains: the motivating question of when and how "perturbed" MCMC chains are developed, the theoretical problem of how perturbation theory can be used to…
This is a review of current research in Markov chains as toric statistical models. Its content is mixture of background information, results from the relevant recent literature, new results, and work in progress.
There are known expressions to calculate the moments of the first passage time in Markov chains. Nevertheless, it is commonly forgotten that in most applications the parameters of the Markov chain are constructed using estimates based upon…
Markov chains for probability distributions related to matrix product states and 1D Hamiltonians are introduced. With appropriate 'inverse temperature' schedules, these chains can be combined into a random approximation scheme for ground…
In this paper we investigate the continuum limits of a class of Markov chains. The investigation of such limits is motivated by the desire to model very large networks. We show that under some conditions, a sequence of Markov chains…
The authors propose a new variation of random walks called ladder chains $L(r,s,p)$. We extend concepts such as ruin probability, hitting time, transience and recurrence of random walks to ladder chain. Take $L(2,2,p)$ for instance, we find…
Given a finite Markov chain, we investigate the first minors of the transition matrix of a lifting of this Markov chain to covering trees. In a simple case we exhibit a nice factorisation of these minors, and we conjecture that it holds…
In this paper we construct (nonhomogeneous) quantum Markov chains associated with open quantum random walks. The quantum Markov chain, like the classical Markov chain, is a fundamental tool for the investigation of the basic properties such…
For a Markov chain $Y$ with values in a Polish space, consider the entrance chain, obtained by sampling $Y$ at the moments when it enters a fixed set $A$ from its complement $A^c$. Similarly, consider the exit chain, obtained by sampling…
This review paper provides an introduction of Markov chains and their convergence rates which is an important and interesting mathematical topic which also has important applications for very widely used Markov chain Monte Carlo (MCMC)…
Random walks cannot, in general, be pushed forward by quasi-isometries. Tame Markov chains were introduced as a `quasi-isometry invariant' are a generalization of random walks. In this paper, we construct several examples of tame Markov…
Probability generating functions for first passage times of Markov chains are found using the method of collective marks. A system of equations is found which can be used to obtain moments of the first passage times.
. Markov chains in time, such as simple random walks, are at the heart of probability. In space, due to the absence of an obvious definition of past and future, a range of definitions of Markovianity have been proposed. In this paper, after…
This paper offers a personal review of some things we've learned about rates of convergence of Markov chains to their stationary distributions. The main topic is ways of speeding up diffusive behavior. It also points to open problems and…
This is an introductory article about Markov Chain Monte Carlo (MCMC) simulation for pedestrians. Actual simulation codes are provided, and necessary practical details, which are skipped in most textbooks, are shown. The second half is…