Related papers: The elapsed time between two transient state obser…
We study the expected accumulated reward for a discrete-time Markov reward model with absorbing states. The rewards are impulse rewards, where a reward $\rho_{ij}$ is accumulated when transitioning from state $i$ to state $j$. We derive an…
In any Markov chain with finite state space the distribution of transition records always belongs to the exponential family. This observation is used to prove a fluctuation theorem, and to show that the dynamical entropy of a stationary…
Multi-state models are frequently applied for representing processes evolving through a discrete set of state. Important classes of multi-state models arise when transitions between states may depend on the time since entry into the current…
A well-known theorem for an irreducible skip-free chain with absorbing state $d$, under some conditions, is that the hitting (absorbing) time of state $d$ starting from state 0 is distributed as the sum of $d$ independent geometric (or…
The time evolution of the thermally activated decay rates is considered. This evolution is of particular importance for the recent nanoscale experiments discussed in the literature, where the potential barrier is relatively low (or the…
In this paper, we are interested in investigating the perturbation bounds for the stationary distributions for discrete-time or continuous-time Markov chains on a countable state space. For discrete-time Markov chains, two new norm-wise…
We characterize absorption-time distributions for birth-death Markov chains with an absorbing boundary. For "extinction-prone" chains (which drift on average toward the absorbing state) the asymptotic distribution is Gaussian, Gumbel, or…
Epidemic dynamics in a stochastic network of interacting epidemic centers is considered. The epidemic and migration processes are modelled by Markov's chains. Explicit formulas for probability distribution of the migration process are…
The dynamics of many epidemic compartmental models for infectious diseases that spread in a single host population present a second-order phase transition. This transition occurs as a function of the infectivity parameter, from the absence…
In this paper, we demonstrate through the use of matrix calculus a transparent analysis of fractional inhomogeneous Markov models for life insurance where transition matrices commute. The resulting formulae are intuitive matrix…
We consider Markov chains with random transition probabilities which, moreover, fluctuate randomly with time. We describe such a system by a product of stochastic matrices, $U(t)=M_t\cdots M_1$, with the factors $M_i$ drawn independently…
We establish an abstract, effective, exponential large deviations type estimate for Markov systems satisfying a weaker form of mixing. We employ this result to derive such estimates, as well as a central limit theorem, for the skew product…
We study Markov chains which model genome rearrangements. These models are useful for studying the equilibrium distribution of chromosomal lengths, and are used in methods for estimating genomic distances. The primary Markov chain studied…
Statistical systems are conceived from the standpoint of statistical mechanics, as made of a (generally large) number of identical units and exhibiting a (generally large) number of different configurations (microstates), among which only…
In many dynamical systems in nature, the law of the dynamics changes along with the temporal evolution of the system. These changes are often associated with the occurrence of certain events. The timing of occurrence of these events…
A Markov state model is a powerful tool that can be used to track the evolution of populations of configurations in an atomistic representation of a protein. For a coarse-grained linear chain model with discontinuous interactions, the…
The Moran process models the spread of mutations in populations on graphs. We investigate the absorption time of the process, which is the time taken for a mutation introduced at a randomly chosen vertex to either spread to the whole…
A general theory is developed to study individual based models which are discrete in time. We begin by constructing a Markov chain model that converges to a one-dimensional map in the infinite population limit. Stochastic fluctuations are…
Problem of adaptive state observer synthesis for linear time-varying (LTV) system with unknown time-varying parameter and delayed output measurements is considered. State observation problem has attracted the attention of many researchers…
The estimation of absorption time distributions of Markov jump processes is an important task in various branches of statistics and applied probability. While the time-homogeneous case is classic, the time-inhomogeneous case has recently…