Related papers: Waiting times and stopping probabilities for patte…
Given an initial (resp., terminal) probability measure $\mu$ (resp., $\nu$) on $\mathbb{R}^d$, we characterize those optimal stopping times $\tau$ that maximize or minimize the functional $\mathbb{E} |B_0 - B_\tau|^{\alpha}$, $\alpha > 0$,…
This paper presents a wp-style calculus for obtaining bounds on the expected run-time of probabilistic programs. Its application includes determining the (possibly infinite) expected termination time of a probabilistic program and proving…
Classical linear regression is considered for a case when regression parameters depend on the external random environment. The last is described as a continuous time Markov chain with finite state space. Here the expected sojourn times in…
We study the problem of identity testing of markov chains. In this setting, we are given access to a single trajectory from a markov chain with unknown transition matrix $Q$ and the goal is to determine whether $Q = P$ for some known matrix…
In this paper, we consider a general class of two-time-scale Markov chains whose transition rate matrix depends on a parameter $\lambda>0$. We assume that some transition rates of the Markov chain will tend to infinity as…
This paper proposes a new type of recurrence where we divide the Markov chains into intervals that start when the chain enters into a subset A, then sample another subset B far away from A and end when the chain again return to A. The…
Let $X_n$ be a discrete time Markov chain with state space $S$ (countably infinite, in general) and initial probability distribution $\mu^{(0)} = (P(X_0=i_1),P(X_0=i_2),\cdots,)$. What is the probability of choosing in random some $k \in…
We use Markov chains and numerical linear algebra -- and several CPU hours -- to determine the expected number of coins in a person's possession under certain conditions. We identify the spending strategy that results in the minimum…
Consider a compact metric space $S$ and a pair $(j,k)$ with $k \ge 2$ and $1 \le j \le k$. For any probability distribution $\theta \in P(S)$, define a Markov chain on $S$ by: from state $s$, take $k$ i.i.d. ($\theta$) samples, and jump to…
We study properties and parameter estimation of finite-state homogeneous continuous-time bivariate Markov chains. Only one of the two processes of the bivariate Markov chain is observable. The general form of the bivariate Markov chain…
Suppose X and Y are two independent irreducible Markov chains on n states. We consider the intersection time, which is the first time their trajectories intersect. We show for reversible and lazy chains that the total variation mixing time…
We ask if it is possible to find some particular continuous paths of unit length in linear Brownian motion. Beginning with a discrete version of the problem, we derive the asymptotics of the expected waiting time for several interesting…
Parametric Markov chains occur quite naturally in various applications: they can be used for a conservative analysis of probabilistic systems (no matter how the parameter is chosen, the system works to specification); they can be used to…
We develop a general theory of the time distribution of quantum events, applicable to a large class of problems such as arrival time, dwell time and tunneling time. A stopwatch ticks until an awaited event is detected, at which time the…
In this paper a method based on a Markov chain Monte Carlo (MCMC) algorithm is proposed to compute the probability of a rare event. The conditional distribution of the underlying process given that the rare event occurs has the probability…
Continuous Time Markov Chain (CMTC) is widely used to describe and analyze systems in several knowledge areas. Steady state availability is one important analysis that can be made through Markov chain formalism that allows researchers…
Define $(X_n)$ on $\mathbf{Z}/q\mathbf{Z}$ by $X_{n+1} = 2X_n + b_n$, where the steps $b_n$ are chosen independently at random from $-1, 0, +1$. The mixing time of this random walk is known to be at most $1.02 \log_2 q$ for almost all odd…
Lyapunov drift and Lyapunov optimization are powerful techniques for optimizing time averages in stochastic queueing networks subject to stability. However, there are various definitions of queue stability in the literature, and the most…
An analytical formula for the occurence probability of Markovian stochastic paths with repeatedly visited and/or equal departure rates is derived. This formula is essential for an efficient investigation of the trajectories belonging to…
Consider a Markov chain $\{X_n\}_{n\ge 0}$ with an ergodic probability measure $\pi$. Let $\Psi$ a function on the state space of the chain, with $\alpha$-tails with respect to $\pi$, $\alpha\in (0,2)$. We find sufficient conditions on the…