Related papers: Waiting times and stopping probabilities for patte…
Random walks (or Markov chains) are models extensively used in theoretical computer science. Several tools, including analysis of quantities such as hitting and mixing times, are helpful for devising randomized algorithms. A notable example…
We show how to construct a topological Markov map of the interval whose invariant probability measure is the stationary law of a given stochastic chain of infinite order. In particular we caracterize the maps corresponding to stochastic…
We present a novel algorithm to solve a non-linear system of equations, whose solution can be interpreted as a tight lower bound on the vector of expected hitting times of a Markov chain whose transition probabilities are only partially…
In the classical theory of Markov chains, one may study the mean time to reach some chosen state, and it is well-known that in the irreducible, finite case, such quantity can be calculated in terms of the fundamental matrix of the walk, as…
Gerber and Li in \cite{GeLi} formulated, using a Markov chain embedding, a system of equations that describes relations between generating functions of waiting time distributions for occurrences of patterns in a sequence of independent…
The mixer chain on a graph G is the following Markov chain. Place tiles on the vertices of G, each tile labeled by its corresponding vertex. A "mixer" moves randomly on the graph, at each step either moving to a randomly chosen neighbor, or…
Markov chain Monte Carlo (MCMC) simulations are commonly employed for estimating features of a target distribution, particularly for Bayesian inference. A fundamental challenge is determining when these simulations should stop. We consider…
A survey of a variety of computational procedures for finding the mean first passage times in Markov chains is presented. The author recently developed a new accurate computational technique, an Extended GTH Procedure, Hunter (Special…
A novel quickest detection setting is proposed which is a generalization of the well-known Bayesian change-point detection model. Suppose \{(X_i,Y_i)\}_{i\geq 1} is a sequence of pairs of random variables, and that S is a stopping time with…
In this paper we develop the elements of the theory of algorithmic randomness in continuous-time Markov chains (CTMCs). Our main contribution is a rigorous, useful notion of what it means for an individual trajectory of a CTMC to be random.…
We consider a model of a population of fixed size N in which each individual gets replaced at rate one and each individual experiences a mutation at rate \mu. We calculate the asymptotic distribution of the time that it takes before there…
A method of constructing Markov chains on finite state spaces is provided. The chain is specified by three constraints: stationarity, dependence and marginal distributions. The generalized Pythagorean theorem in information geometry plays a…
We address the problem of community detection in networks by introducing a general definition of Markov stability, based on the difference between the probability fluxes of a Markov chain on the network at different time scales. The…
Let $LA_{n}(\tau)$ be the length of the longest alternating subsequence of a uniform random permutation $\tau\in[n]$. Classical probabilistic arguments are used to rederive the asymptotic mean, variance and limiting law of $LA_{n}(\tau)$.…
In this paper we consider Markov chains with transition rates that depend on a small parameter $\varepsilon$. Under a mild assumption on the asymptotics of these transition rates, we describe the behavior of the chain at various…
In the standard models for optimal multiple stopping problems it is assumed that between two exercises there is always a time period of deterministic length $\delta$, the so called refraction period. This prevents the optimal exercise times…
We consider continuous-space, discrete-time Markov chains on $\mathbb{R}^d$, that admit a finite number $N$ of metastable states. Our main motivation for investigating these processes is to analyse random Poincar\'e maps, which describe…
This paper focuses on the problem of modeling the correspondence pattern for ordinary people. Suppose that letters arrive at a rate $\lambda$ and are answered at a rate $\mu$. Furthermore, we assume that, for a constant $T$, a letter is…
Polling systems are a well-established subject in queueing theory. However, their formal treatments generally rely heavily on relatively sophisticated theoretical tools, such as moment generating functions and Laplace transforms, and…
This article shows how coupled Markov chains that meet exactly after a random number of iterations can be used to generate unbiased estimators of the solutions of the Poisson equation. Through this connection, we re-derive known unbiased…