Related papers: The Entropy of a Binary Hidden Markov Process
We investigate stationary hidden Markov processes for which mutual information between the past and the future is infinite. It is assumed that the number of observable states is finite and the number of hidden states is countably infinite.…
We study the entropy rate of pattern sequences of stochastic processes, and its relationship to the entropy rate of the original process. We give a complete characterization of this relationship for i.i.d. processes over arbitrary…
We establish upper and lower bounds for the metric entropy and bracketing entropy of the class of $d$-dimensional bounded monotonic functions under $L^p$ norms. It is interesting to see that both the metric entropy and bracketing entropy…
Channels with noisy duplications have recently been used to model the nanopore sequencer. This paper extends some foundational information-theoretic results to this new scenario. We prove the asymptotic equipartition property (AEP) for…
The Shannon entropy, and related quantities such as mutual information, can be used to quantify uncertainty and relevance. However, in practice, it can be difficult to compute these quantities for arbitrary probability distributions,…
The generic identification problem is to decide whether a stochastic process $(X_t)$ is a hidden Markov process and if yes to infer its parameters for all but a subset of parametrizations that form a lower-dimensional subvariety in…
Characterizing the entropy of a system is a crucial, and often computationally costly, step in understanding its thermodynamics. It plays a key role in the study of phase transitions, pattern formation, protein folding and more. Current…
The complete framework for the $\epsilon$-machine construction of the one dimensional Ising model is presented correcting previous mistakes on the subject. The approach follows the known treatment of the Ising model as a Markov random…
A new method is presented which allows time averaged density matrices of closed quantum systems to be computed via a constraint overlap maximization. Due to its simplicity, this method can be combined with algorithms based on tensor…
A method for correcting for detector smearing effects using machine learning techniques is presented. Compared to the standard approaches the method can use more than one reconstructed variable to infere the value of the unsmeared quantity…
By establishing Multiplicative Ergodic Theorem for commutative transformations on a separable infinite dimensional Hilbert space, in this paper, we investigate Pesin's entropy formula and SRB measures of a finitely generated random…
We investigate entropy production for an expanding system of particles and resonances with isospin symmetry -- in our case pions and $\rho$ mesons -- within the framework of relativistic kinetic theory. A cascade code to simulate the…
Entropy is a measure of self-information which is used to quantify losses. Entropy was developed in thermodynamics, but is also used to compare probabilities based on their deviating information content. Corresponding model uncertainty is…
Black hole entropy is studied for an exactly solvable model of two-dimensional gravity\cite{rst1}, using recently developed Noether charge techniques\cite{wald1}. This latter approach is extended to accomodate the non-local form of the…
We present two examples of finite-alphabet, infinite excess entropy processes generated by invariant hidden Markov models (HMMs) with countable state sets. The first, simpler example is not ergodic, but the second is. It appears these are…
An autoregressive process with Markov regime is an autoregressive process for which the regression function at each time point is given by a nonobservable Markov chain. In this paper we consider the asymptotic properties of the maximum…
We study irreversible processes for nonlinear oscillators networks described by complex-valued Langevin equations that account for coupling to different thermo-chemical baths. Dissipation is introduced via non-Hermitian terms in the…
We establish a sharp estimate for a minimal number of binary digits (bits) needed to represent all bounded total generalized variation functions taking values in a general totally bounded metric space $(E,\rho)$ up to an accuracy of…
The purpose of this note is to show how the method of maximum entropy in the mean (MEM) may be used to improve parametric estimation when the measurements are corrupted by large level of noise. The method is developed in the context on a…
There is a relation between the irreversibility of thermodynamic processes as expressed by the breaking of time-reversal symmetry, and the entropy production in such processes. We explain on an elementary mathematical level the relations…