Related papers: Entropy of absorbed Markov chains
Symbolic dynamics has proven to be an invaluable tool in analyzing the mechanisms that lead to unpredictability and random behavior in nonlinear dynamical systems. Surprisingly, a discrete partition of continuous state space can produce a…
We formulate some simple conditions under which a Markov chain may be approximated by the solution to a differential equation, with quantifiable error probabilities. The role of a choice of coordinate functions for the Markov chain is…
In this paper, we give a sufficient condition for the existence of a quasi-ergodic distribution for absorbing Markov processes. Using an orthogonal-polynomial approach, we prove that the previous main result is valid for the birth-death…
The embedding problem for Markov chains is a famous problem in probability theory and only partial results are available up till now. In this paper, we propose a variant of the embedding problem called the reversible embedding problem which…
Maximum entropy (maxEnt) inference of state probabilities using state-dependent constraints is popular in the study of complex systems. In stochastic dynamical systems, the effect of state space topology and path-dependent constraints on…
Motivated by robotic surveillance applications, this paper studies the novel problem of maximizing the return time entropy of a Markov chain, subject to a graph topology with travel times and stationary distribution. The return time entropy…
We prove that under mild positivity assumptions the entropy rate of a hidden Markov chain varies analytically as a function of the underlying Markov chain parameters. A general principle to determine the domain of analyticity is stated. An…
We establish sufficient conditions for exponential convergence to a unique quasi-stationary distribution in the total variation norm. These conditions also ensure the existence and exponential ergodicity of the Q-process, the process…
Estimating the entropy based on data is one of the prototypical problems in distribution property testing and estimation. For estimating the Shannon entropy of a distribution on $S$ elements with independent samples, [Paninski2004] showed…
We present two new methods for estimating the order (memory depth) of a finite alphabet Markov chain from observation of a sample path. One method is based on entropy estimation via recurrence times of patterns, and the other relies on a…
We propose the entropy of random Markov trajectories originating and terminating at a state as a measure of the stability of a state of a Markov process. These entropies can be computed in terms of the entropy rates and stationary…
Stochastic chains represent a wide and key variety of phenomena in many branches of science within the context of Information Theory and Thermodynamics. They are typically approached by a sequence of independent events or by a memoryless…
The correspondence between the cross-entropy method and the zero-variance approximation to simulate a rare event problem in Markov chains is shown. This leads to a sufficient condition that the cross-entropy estimator is asymptotically…
Computing the stationary distributions of a continuous-time Markov chain (CTMC) involves solving a set of linear equations. In most cases of interest, the number of equations is infinite or too large, and the equations cannot be solved…
We consider continuous-space, discrete-time Markov chains on $\mathbb{R}^d$, that admit a finite number $N$ of metastable states. Our main motivation for investigating these processes is to analyse random Poincar\'e maps, which describe…
We consider the Markov chain approximations for singular stable-like processes. First we obtain properties of some Markov chains. Then we construct the approximating Markov chains and give a necessary condition for weak convergence of these…
Epidemic dynamics in a stochastic network of interacting epidemic centers is considered. The epidemic and migration processes are modelled by Markov's chains. Explicit formulas for probability distribution of the migration process are…
We consider a simple but important class of metastable discrete time Markov chains, which we call perturbed Markov chains. Basically, we assume that the transition matrices depend on a parameter $\varepsilon$, and converge as $\varepsilon$.…
It has recently been shown that structural conditions on the reaction network, rather than a 'fine-tuning' of system parameters, often suffice to impart 'absolute concentration robustness' on a wide class of biologically relevant,…
Substitution Markov chains have been introduced [7] as a new model to describe molecular evolution. In this note, we study the associated Martin boundaries from a probabilistic and topological viewpoint. An example is given that, although…