Related papers: Entropy Rate for Hidden Markov Chains with rare tr…
We use the $f-divergence$ also called relative entropy as a measure of diversity between probability densities and review its basic properties. In the sequence we define a few objects which capture relevant information from the sample of a…
A time-dependent finite-state Markov chain that uses doubly stochastic transition matrices, is considered. Entropic quantities that describe the randomness of the probability vectors, and also the randomness of the discrete paths, are…
We consider the convergence of a continuous-time Markov chain approximation X^h, h>0, to an R^d-valued Levy process X. The state space of X^h is an equidistant lattice and its Q-matrix is chosen to approximate the generator of X. In…
In this paper, we develop methods of nonlinear filtering and prediction of an unobservable Markov chain with a finite set of states. This Markov chain controls coefficients of AR(p) model. Using observations generated by AR(p) model we have…
We consider the parameter estimation of Markov chain when the unknown transition matrix belongs to an exponential family of transition matrices. Then, we show that the sample mean of the generator of the exponential family is an…
We derive a novel chain rule for a family of channel conditional entropies, covering von Neumann and sandwiched R\'{e}nyi entropies. In the process, we show that these channel conditional entropies are equal to their regularized version,…
Given a unichain Markov reward process (MRP), we provide an explicit expression for the bias values in terms of mean first passage times. This result implies a generalization of known Markov chain perturbation bounds for the stationary…
For Markov chains and Markov processes exhibiting a form of stochastic monotonicity (larger states shift up transition probabilities in terms of stochastic dominance), stability and ergodicity results can be obtained using order-theoretic…
We give computable bounds on the rate of convergence of the transition probabilities to the stationary distribution for a certain class of geometrically ergodic Markov chains. Our results are different from earlier estimates of Meyn and…
The paper studies an improved estimate for the rate of convergence for nonlinear homogeneous discrete-time Markov chains. These processes are nonlinear in terms of the distribution law. Hence, the transition kernels are dependent on the…
We present a closed-form, computable expression for the expected number of times any transition event occurs during the transient phase of a reducible Markov chain. Examples of events include time to absorption, number of visits to a state,…
Kemeny's constant measures the efficiency of a Markov chain in traversing its states. We investigate whether structure-preserving perturbations to the transition probabilities of a reversible Markov chain can improve its connectivity while…
Rate processes are simple and analytically tractable models for many dynamical systems which switch stochastically between a discrete set of quasi stationary states but they may also approximate continuous processes by coarse grained,…
Stochastic reaction networks are mathematical models frequently used in, but not limited to, biochemistry. These models are continuous-time Markov chains whose transition rates depend on certain parameters called rate constants, which…
We consider a stochastic process which is (a) described by a continuous-time Markov chain on only short time-scales and (b) constrained to conserve a number of hidden quantities on long time-scales. We assume that the transition matrix of…
A transition matrix can be constructed through the partial contraction of two given quantum states. We analyze and compare four different definitions of entropy for transition matrices, including (modified) pseudo entropy, SVD entropy, and…
Using the concept of discrete noiseless channels, it was shown by Shannon in A Mathematical Theory of Communication that the ultimate performance of an encoder for a constrained system is limited by the combinatorial capacity of the system…
We study inhomogeneous continuous-time weakly ergodic Markov chains with a finite state space. We introduce the notion of a Markov chain with the regular structure of an infinitesimal matrix and study the sharp upper bounds on the rate of…
In this paper we study ergodic theory of countable Markov shifts. These are dynamical systems defined over non-compact spaces. Our main result relates the escape of mass, the measure theoretic entropy, and the entropy at infinity of the…
Spiking activity from populations of neurons display causal interactions and memory effects. Therefore, they are expected to show some degree of irreversibility in time. Motivated by the spike train statistics, in this paper we build a…