Related papers: Kemeny's Constant for Markov Processes
Consider a Markov chain with finite state $\{0, 1, ..., d\}$. We give the generation functions (or Laplace transforms) of absorbing (passage) time in the following two situations : (1) the absorbing time of state $d$ when the chain starts…
We consider the computational task of sampling a bit string $x$ from a distribution $\pi(x)=|\langle x|\psi\rangle|^2$, where $\psi$ is the unique ground state of a local Hamiltonian $H$. Our main result describes a direct link between the…
We obtain universal estimates on the convergence to equilibrium and the times of coupling for continuous time irreducible reversible finite-state Markov chains, both in the total variation and in the L^2 norms. The estimates in total…
We consider a discrete time hidden Markov model where the signal is a stationary Markov chain. When conditioned on the observations, the signal is a Markov chain in a random environment under the conditional measure. It is shown that this…
In a recent paper, Shah [arXiv:2502.03073] derived an explicit expression for the distribution of occupancy times for a two-state Markov chain, using a method based on enumerating sample paths. We consider here the more general problem of…
The hitting and mixing times are two fundamental quantities associated with Markov chains. In Peres and Sousi[PS2015] and Oliveira[Oli2012], the authors show that the mixing times and "worst-case" hitting times of reversible Markov chains…
In this brief note, we find formulas for the distribution and the transition probability matrices of a stochastic process described as a time-reversion in a finite time window of a Markov chain, with cluster observation of the Markov state…
Here, a new two-dimensional process, discrete in time and space, that yields the results of both a random walk and a quantum random walk, is introduced. This model describes the population distribution of four coin states |1>,-|1>, |0> -|0>…
It is shown that irreducible two-state continuous-time Markov chains interacting on a network in a bilinear fashion have a unique stable steady state. The proof is elementary and uses the relative entropy function.
All one-condition generalized inverses of the Markovian kernel I - P, where P is the transition matrix of a finite irreducible Markov chain, can be uniquely specified in terms of the stationary probabilities and the mean first passage times…
We propose a new approach for estimating the finite dimensional transition matrix of a Markov chain using a large number of independent sample paths observed at random times. The sample paths may be observed as few as two times, and the…
Scaled type Markov renewal processes generalize classical renewal processes: renewal times come from a one parameter family of probability laws and the sequence of the parameters is the trajectory of an ergodic Markov chain. Our primary…
In the absence of acceleration, the velocity formula gives "distance travelled equals speed multiplied by time". For a broad class of Markov chains such as circulant Markov chains or random walk on complete graphs, we prove a probabilistic…
Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…
Measuring entropy production of a system directly from the experimental data is highly desirable since it gives a quantifiable measure of the time-irreversibility for non-equilibrium systems and can be used as a cost function to optimize…
The entropy production rate is central to the study of non-equilibrium systems. This parameter is closely connected to violation of time-reversal symmetry, energy consumption, efficiency, and other properties of interest; in short, it…
We consider finite-state time-nonhomogeneous Markov chains where the probability of moving from state $i$ to state $j\neq i$ at time $n$ is $G(i,j)/n^\zeta$ for a ``generator'' matrix $G$ and strength parameter $\zeta>0$. In these chains,…
We propose a method to approximate continuous-time, continuous-state stochastic processes by a discrete-time Markov chain defined on a nonuniform grid. Our method provides exact moment matching for processes whose first and second moments…
The purpose of this paper is to study the time average behavior of Markov chains with transition probabilities being kernels of completely continuous operators, and therefore to provide a sufficient condition for a class of Markov chains…
The parameters of a discrete stationary Markov model are transition probabilities between states. Traditionally, data consist in sequences of observed states for a given number of individuals over the whole observation period. In such a…