Related papers: Waiting times and stopping probabilities for patte…
We present here a general framework and a specific algorithm for predicting the destination, route, or more generally a pattern, of an ongoing journey, building on the recent work of [Y. Lassoued, J. Monteil, Y. Gu, G. Russo, R. Shorten,…
Given a possibly discontinuous, bounded function $f:\mathbb{R}\mapsto\mathbb{R}$, we consider the set of generalized flows, obtained by assigning a probability measure on the set of Carath\'eodory solutions to the ODE ~$\dot x = f(x)$. The…
In the present paper, we investigate the relationship between hitting times and hitting probabilities in discrete-time imprecise Markov chains (IMCs). We define lower and upper hitting times and probabilities for IMCs whose set of…
Consider a Markov chain $(X_n)_{n\geqslant 0}$ with values in the state space $\mathbb X$. Let $f$ be a real function on $\mathbb X$ and set $S_0=0,$ $S_n = f(X_1)+\cdots + f(X_n),$ $n\geqslant 1$. Let $\mathbb P_x$ be the probability…
Let $P$ be a bistochastic matrix of size $n$, and let $\Pi$ be a permutation matrix of size $n$. In this paper, we are interested in the mixing time of the Markov chain whose transition matrix is given by $Q=P\Pi$. In other words, the chain…
We study convergence to equilibrium for a large class of Markov chains in random environment. The chains are sparse in the sense that in every row of the transition matrix $P$ the mass is essentially concentrated on few entries. Moreover,…
The convergence rate of a Markov chain to its stationary distribution is typically assessed using the concept of total variation mixing time. However, this worst-case measure often yields pessimistic estimates and is challenging to infer…
We investigate the effects of markovian resseting events on continuous time random walks where the waiting times and the jump lengths are random variables distributed according to power law probability density functions. We prove the…
To quantify the randomness of Markov trajectories with fixed initial and final states, Ekroot and Cover proposed a closed-form expression for the entropy of trajectories of an irreducible finite state Markov chain. Numerous applications,…
Let $\xi_1,\xi_2,\ldots$ be independent, identically distributed random variables with infinite mean $\mathbf E[|\xi_1|]=\infty.$ Consider a random walk $S_n=\xi_1+\cdots+\xi_n$, a stopping time $\tau=\min\{n\ge 1: S_n\le 0\}$ and let…
It is well-known that 0 is the absorbing state for a branching system. Each particle in the system lives a random long time and gives a random number of new particles at its death time. It stops when the system has no particle. This paper…
We consider Markov processes, which describe e.g. queueing network processes, in a random environment which influences the network by determining random breakdown of nodes, and the necessity of repair thereafter. Starting from an explicit…
We propose a new approach for estimating the finite dimensional transition matrix of a Markov chain using a large number of independent sample paths observed at random times. The sample paths may be observed as few as two times, and the…
Starting from a Markov chain with a finite alphabet, we consider the chain obtained when all but one symbol are undistinguishable for the practitioner. We study necessary and sufficient conditions for this chain to have continuous…
We study continuous time Markov processes on graphs. The notion of frequency is introduced, which serves well as a scaling factor between any Markov time of a continuous time Markov process and that of its jump chain. As an application, we…
We model the scrambling of a Rubik's cube by a Markov chain and introduce a stopping time $T$ which is a quite natural candidate to be a strong uniform time. This may pave the way for estimating the number of moves required to scramble a…
We describe a new construction of a family of measures on a group with the same Poisson boundary. Our approach is based on applying Markov stopping times to an extension of the original random walk.
A cornerstone of human statistical learning is the ability to extract temporal regularities / patterns from random sequences. Here we present a method of computing pattern time statistics with generating functions for first-order Markov…
The paper addresses the problem of computing maximal conditional expected accumulated rewards until reaching a target state (briefly called maximal conditional expectations) in finite-state Markov decision processes where the condition is…
We investigate the hitting times of random walks on graphs, where a hitting time is defined as the number of steps required for a random walker to move from one node to another. While much of the existing literature focuses on calculating…