Related papers: The Entropy of a Binary Hidden Markov Process

We consider Hidden Markov Chains obtained by passing a Markov Chain with rare transitions through a noisy memoryless channel. We obtain asymptotic estimates for the entropy of the resulting Hidden Markov Chain as the transition rate is…

Entropy in nonequilibrium statistical mechanics is investigated theoretically so as to extend the well-established equilibrium framework to open nonequilibrium systems. We first derive a microscopic expression of nonequilibrium entropy for…

This paper considers the derivative of the entropy rate of a hidden Markov process with respect to the observation probabilities. The main result is a compact formula for the derivative that can be evaluated easily using Monte Carlo…

Entropy estimation, due in part to its connection with mutual information, has seen considerable use in the study of time series data including causality detection and information flow. In many cases, the entropy is estimated using…

In this paper, we introduce the notion of Bi-entangled hidden Markov processes. These are hidden quantum processes where the hidden processes themselves exhibit entangled Markov process, and the observable processes also exhibit…

Inference in hidden Markov model has been challenging in terms of scalability due to dependencies in the observation data. In this paper, we utilize the inherent memory decay in hidden Markov models, such that the forward and backward…

We investigate the performance of entropy estimation methods, based either on block entropies or compression approaches, in the case of bidimensional sequences. We introduce a validation dataset made of images produced by a large number of…

Determining entropy rates of stochastic processes is a fundamental and difficult problem, with closed-form solutions known only for specific cases. This paper pushes the state-of-the-art by solving the problem for Hidden Markov Models…

A seven parameter family of five-dimensional black hole solutions depending on mass, two angular momenta, three charges and the asymptotic value of a scalar field is constructed. The entropy is computed as a function of these parameters…

The analyticity of the entropy and relative entropy rates of continuous-state hidden Markov models is studied here. Using the analytic continuation principle and the stability properties of the optimal filter, the analyticity of these rates…

We study the multiple definitions of the entropy production for discrete-time Markov processes in single systems and composite systems. These definitions have been studied in single systems, but less so in composite systems. With a clear…

Entropy production (EP) is a central measure in nonequilibrium thermodynamics, as it can quantify the irreversibility of a process as well as its energy dissipation in special cases. Using the time-reversal asymmetry in a system's path…

We investigate a stationary process's crypticity---a measure of the difference between its hidden state information and its observed information---using the causal states of computational mechanics. Here, we motivate crypticity and cryptic…

The main goal of this paper is to develop an estimate for the entropy of random stationary ergodic symbolic sequences with elements belonging to a finite alphabet. We present here the detailed analytical study of the entropy for the…

Entropy functionals are computed for non-stationary distributions of particles of Lorentz gas and hard disks. The distributions consisting of beams of particles are found to have the largest amount of entropy and entropy increase. The…

Given a sequence of observations from a discrete-time, finite-state hidden Markov model, we would like to estimate the sampling distribution of a statistic. The bootstrap method is employed to approximate the confidence regions of a…

We prove that under mild positivity assumptions the entropy rate of a hidden Markov chain varies analytically as a function of the underlying Markov chain parameters. A general principle to determine the domain of analyticity is stated. An…

Mrs. Gerber's Lemma lower bounds the entropy at the output of a binary symmetric channel in terms of the entropy of the input process. In this paper, we lower bound the output entropy via a different measure of input uncertainty, pertaining…

Transfer entropy is a measure of the magnitude and the direction of information flow between jointly distributed stochastic processes. In recent years, its permutation analogues are considered in the literature to estimate the transfer…

We define the projected entropy S(T) at a given temperature T in the context of an Ising model transition matrix calculation as the entropy associated with the distribution of Markov chain realizations in energy-magnetization, E-H, space.…