Related papers: Entropy Rate for Hidden Markov Chains with rare tr…
We are interested in the analysis of very large continuous-time Markov chains (CTMCs) with many distinct rates. Such models arise naturally in the context of reliability analysis, e.g., of computer network performability analysis, of power…
We study the N-step binary stationary ergodic Markov chain and analyze its differential entropy. Supposing that the correlations are weak we express the conditional probability function of the chain through the pair correlation function and…
In this paper a new estimator for the transition density $\pi$ of an homogeneous Markov chain is considered. We introduce an original contrast derived from regression framework and we use a model selection method to estimate $\pi$ under…
A hidden Markov model (HMM) is said to have path-mergeable states if for any two states i,j there exists a word w and state k such that it is possible to transition from both i and j to k while emitting w. We show that for a finite HMM with…
In this work we study the asymptotic of renewal sequences associated with certain transient renewal Markov chains and enquire about the existence of limit laws in this set up.
Many years ago B.S. Pitskel observed that the metric entropy of the shift transformation in the sample space of a stationary random process $X=\{X_n,\,n\in \mathbb Z\}$ with a countable number of states is equal to the conditional entropy…
We present an algorithm for learning mixtures of Markov chains and Markov decision processes (MDPs) from short unlabeled trajectories. Specifically, our method handles mixtures of Markov chains with optional control input by going through a…
We propose polynomial-time algorithms to minimise labelled Markov chains whose transition probabilities are not known exactly, have been perturbed, or can only be obtained by sampling. Our algorithms are based on a new notion of an…
We study a class of dynamical systems generated by random substitutions, which contains both intrinsically ergodic systems and instances with several measures of maximal entropy. In this class, we show that the measures of maximal entropy…
We investigate the mixing properties of a model of reversible Markov chains in random environment, which notably contains the simple random walk on the superposition of a deterministic graph and a second graph whose vertex set has been…
In the investigation of limits of Markov chains, the presence of states which become instantaneous states in the limit may prevent the convergence of the chain in the Skorohod topology. We present in this article a weaker topology adapted…
Asymptotic properties of Markov Processes, such as steady state probabilities or hazard rate for absorbing states can be efficiently calculated by means of linear algebra even for large-scale problems. This paper discusses the methods for…
For discrete-time stochastic processes, there is a close connection between return/waiting times and entropy. Such a connection cannot be straightforwardly extended to the continuous-time setting. Contrarily to the discrete-time case one…
For a Markovian dynamics on discrete states, the logarithmic ratio of waiting-time distributions between two successive, instantaneous transitions in forward and backward direction is a measure of time-irreversibility. It thus serves as an…
Even simply-defined, finite-state generators produce stochastic processes that require tracking an uncountable infinity of probabilistic features for optimal prediction. For processes generated by hidden Markov chains the consequences are…
Continuous-time Markov chains are used to model stochastic systems where transitions can occur at irregular times, e.g., birth-death processes, chemical reaction networks, population dynamics, and gene regulatory networks. We develop a…
This paper contributes an in-depth study of properties of continuous time Markov chains (CTMCs) on non-negative integer lattices $\N_0^d$, with particular interest in one-dimensional CTMCs with polynomial transitions rates. Such stochastic…
We study random walk on complex networks with transition probabilities which depend on the current and previously visited nodes. By using an absorbing Markov chain we derive an exact expression for the mean first passage time between pairs…
A randomized algorithm for finding sparse cuts is given which is based on constructing a dual markov chain called multiscale rings process(MRP) and a new concept of entropy. It is shown how the time to absorption of the dual process…
We consider a unified framework of sequential change-point detection and hypothesis testing modeled by means of hidden Markov chains. One observes a sequence of random variables whose distributions are functionals of a hidden Markov chain.…