Related papers: Random walks colliding before getting trapped
We have developed a steady state theory of complex transport networks used to model the flow of commodity, information, viruses, opinions, or traffic. Our approach is based on the use of the Markov chains defined on the graph…
The mixer chain on a graph G is the following Markov chain. Place tiles on the vertices of G, each tile labeled by its corresponding vertex. A "mixer" moves randomly on the graph, at each step either moving to a randomly chosen neighbor, or…
Motivated by robotic surveillance applications, this paper studies the novel problem of maximizing the return time entropy of a Markov chain, subject to a graph topology with travel times and stationary distribution. The return time entropy…
We study distributions of meeting times for finite symmetric Markov chains. For Markov kernels defined on large state spaces which satisfy certain weak inhomogeneity in return probabilities of points up to large numbers of steps, we obtain…
Perturbation theory for Markov chains addresses the question how small differences in the transitions of Markov chains are reflected in differences between their distributions. We prove powerful and flexible bounds on the distance of the…
We consider a continuous-time Markov chain with a finite or countable state space. For a site y and subset H of the state space, the hitting time of y under taboo H is defined to be infinite if the process trajectory hits H before y, and…
We identify a fundamental phenomenon of heterogeneous one dimensional random walks: the escape (traversal) time is maximized when the heterogeneity in transition probabilities forms a pyramid-like potential barrier. This barrier corresponds…
In this paper, we present a novel iterative Monte Carlo method for approximating the stationary probability of a single state of a positive recurrent Markov chain. We utilize the characterization that the stationary probability of a state…
We prove an upper bound on the total variation mixing time of a finite Markov chain in terms of the absolute spectral gap and the number of elements in the state space. Unlike results requiring reversibility or irreducibility, this bound is…
We present a data-driven model predictive control scheme for chance-constrained Markovian switching systems with unknown switching probabilities. Using samples of the underlying Markov chain, ambiguity sets of transition probabilities are…
Given an infinite connected regular graph $G=(V,E)$, place at each vertex Pois($\lambda$) walkers performing independent lazy simple random walks on $G$ simultaneously. When two walkers visit the same vertex at the same time they are…
Random walks on simple graphs in connection with electrical resistor networks lead to the definition of Markov chains with transition probability matrix in terms of electrical conductances. We extend this definition to an effective…
Transition Path Theory (TPT) provides a rigorous statistical characterization of the ensemble of trajectories connecting directly, i.e., without detours, two disconnected (sets of) states in a Markov chain, a stochastic process that…
We present a novel method for computing reachability probabilities of parametric discrete-time Markov chains whose transition probabilities are fractions of polynomials over a set of parameters. Our algorithm is based on two key…
Let $G$ be a connected semisimple real Lie group with finite center, and $\mu$ a probability measure on $G$ whose support generates a Zariski-dense subgroup of $G$. We consider the right $\mu$-random walk on $G$ and show that each random…
For a symmetric, homogeneous and irreducible random walk on d-dimensional integer lattice Z^d, having zero mean and a finite variance of jumps, we study the passage times (with possible infinite values) determined by the starting point x,…
This paper is a variation on the uniform spanning tree theme. We use random spanning forests to solve the following problem: for a Markov process on a finite set of size $n$, find a probability law on the subsets of any given size $m \leq…
Suitable reachability conditions can make two different fixed point semantics of a transition system coincide. For instance, the total and partial expected reward semantics on Markov chains (MCs) coincide whenever the MC at hand is almost…
We show that the convergence of finite state space Markov chains to stationarity can often be considerably speeded up by alternating every step of the chain with a deterministic move. Under fairly general conditions, we show that not only…
We consider a model of random walk in ${\mathbb Z}^2$ with (fixed or random) orientation of the horizontal lines (layers) and with non constant iid probability to stay on these lines. We prove the transience of the walk for any fixed…