Related papers: Approximation of the Exit Probability of a Stable …

Let $(X_n)_{n\ge 1}$ be a Markov chain on a measurable state space $X$, and let $S_n = \sum_{k=1}^n f(X_k)$ be the associated Markov walk. For $y>0$, denote by $\tau_y$ the first time at which $y+S_n$ becomes non-positive. Assuming that the…

Let $(X_n)_{n\geq 0}$ be a Markov chain with values in a finite state space $\mathbb X$ starting at $X_0=x \in \mathbb X$ and let $f$ be a real function defined on $\mathbb X$. Set $S_n=\sum_{k=1}^{n} f(X_k)$, $n\geqslant 1$. For any $y \in…

Let $\{X_n\}$ be a Markov chain with transition probability $p_{ij}=a_{j-(i-1)^+},\forall i,j\ge 0$, where $a_j=0$ provided $j<0$, $a_0>0$, $a_0+a_1<1$ and $\sum_{n=0}^\infty a_n=1$. Let $\mu=\sum_{n=1}^\infty na_n$. It's known that…

We investigate the statistics of the gap, G_n, between the two rightmost positions of a Markovian one-dimensional random walker (RW) after n time steps and of the duration, L_n, which separates the occurrence of these two extremal…

We study the asymptotic behaviour of Markov chains $(X_n,\eta_n)$ on $\mathbb{Z}_+ \times S$, where $\mathbb{Z}_+$ is the non-negative integers and $S$ is a finite set. Neither coordinate is assumed to be Markov. We assume a moments bound…

We consider the problem of detecting a random walk on a graph, based on observations of the graph nodes. When visited by the walk, each node of the graph observes a signal of elevated mean, which we assume can be different across different…

Recently, in ["The coin-turning walk and its scaling limit", Electronic Journal of Probability, 25 (2020)], the ``coin-turning walk'' was introduced on ${\mathbb Z}$. It is a non-Markovian process where the steps form a (possibly)…

Approximating the stationary probability of a state in a Markov chain through Markov chain Monte Carlo techniques is, in general, inefficient. Standard random walk approaches require $\tilde{O}(\tau/\pi(v))$ operations to approximate the…

In the first part of this thesis, we study a Markov chain on $\mathbb{R}_+ \times S$, where $\mathbb{R}_+$ is the non-negative real numbers and $S$ is a finite set, in which when the $\mathbb{R}_+$-coordinate is large, the $S$-coordinate of…

Let $\{\boldsymbol{X}_n\}$ be a discrete-time $d$-dimensional process on $\mathbb{Z}_+^d$ with a supplemental (background) process $\{J_n\}$ on a finite set and assume the joint process $\{\boldsymbol{Y}_n\}=\{(\boldsymbol{X}_n,J_n)\}$ to…

We are interested in the asymptotic behavior of Markov chains on the set of positive integers for which, loosely speaking, large jumps are rare and occur at a rate that behaves like a negative power of the current state, and such that small…

We consider a two-node queue modeled as a two-dimensional random walk. In particular, we consider the case that one or both queues have finite buffers. We develop an approximation scheme based on the Markov reward approach to error bounds…

We consider the first exit time of a nonnegative Harris-recurrent Markov process from the interval $[0,A]$ as $A\to\infty$. We provide an alternative method of proof of asymptotic exponentiality of the first exit time (suitably…

We outline basic properties of a symmetric random walk in one dimension, in which the length of the nth step equals lambda^n, with lambda<1. As the number of steps N-->oo, the probability that the endpoint is at x, P_{lambda}(x;N),…

Consider the batch-arrival $GI^X/M/c/N$ model with $c$ servers, general inter-arrival batch times, finite buffer, and exponential service times. Inter-arrival batch times, batch sizes, and service times are $i.i.d.$ and independent of each…

We study continuous time Markov processes on graphs. The notion of frequency is introduced, which serves well as a scaling factor between any Markov time of a continuous time Markov process and that of its jump chain. As an application, we…

We consider a random walk among i.i.d. obstacles on the one dimensional integer lattice under the condition that the walk starts from the origin and reaches a remote location y. The obstacles are represented by a killing potential, which…

A possibly time-dependent transition intensity matrix or generator $(Q(t))$ characterizes the law of a Markov jump process (MP). For a time homogeneous MP, the transition probability matrix (TPM) can be expressed as a matrix exponential of…

We consider a walker moving in a one-dimensional interval with absorbing boundaries under the effect of Markovian resettings to the initial position. The walker's motion follows a random walk characterized by a general waiting time…

In this paper, we present a numerical framework for constructing bounds on stationary performance measures of random walks in the positive orthant using the Markov reward approach. These bounds are established in terms of stationary…