Related papers: Mixing, Ergodic, and Nonergodic Processes with Rap…
We investigate stationary hidden Markov processes for which mutual information between the past and the future is infinite. It is assumed that the number of observable states is finite and the number of hidden states is countably infinite.…
Shannon's mutual information of a random multiple antenna and multipath channel is studied in the general case where the channel impulse response is an ergodic and stationary process which is assumed to be available at the receiver. From…
We develop operator renewal theory for flows and apply this to infinite ergodic theory. In particular we obtain results on mixing for a large class of infinite measure semiflows. Examples of systems covered by our results include…
Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy…
We develop information theory for the temporal behavior of memoryful agents moving through complex -- structured, stochastic -- environments. We introduce and explore information processes -- stochastic processes produced by cognitive…
We introduce a simple analysis of the structural complexity of infinite-memory processes built from random samples of stationary, ergodic finite-memory component processes. Such processes are familiar from the well known multi-arm Bandit…
We propose strongly consistent estimators of the $\ell_1$ norm of the sequence of $\alpha$-mixing (respectively $\beta$-mixing) coefficients of a stationary ergodic process. We further provide strongly consistent estimators of individual…
We propose methods to estimate the individual $\beta$-mixing coefficients of a real-valued geometrically ergodic Markov process from a single sample-path $X_0,X_1, \dots,X_n$. Under standard smoothness conditions on the densities, namely,…
We study the entropy rate of pattern sequences of stochastic processes, and its relationship to the entropy rate of the original process. We give a complete characterization of this relationship for i.i.d. processes over arbitrary…
The paper concerns the rates of power-law growth of mutual information computed for a stationary measure or for a universal code. The rates are called Hilberg exponents and four such quantities are defined for each measure and each code:…
We address the excess entropy, which is a measure of complexity for stationary time series, from the ordinal point of view. We show that the permutation excess entropy is equal to the mutual information between two adjacent semi-infinite…
The application of the Shannon entropy to study the relationship between information and structures has yielded insights into molecular and material systems. However, the difficulty in directly observing and manipulating atoms and molecules…
The paper is devoted to equipartition of measured information for finite state processes over regular trees whose laws are invariant under all parity preserving tree automorphisms. We show almost everywhere equipartition for ergodic…
Entropies of mixing can be derived directly from the parent distributions of extreme value theory. They correspond to pseudo-additive entropies in the case of Pareto and power function distributions, while to the Shannon entropy in the case…
We prove that a stationary max--infinitely divisible process is mixing (ergodic) iff its dependence function converges to 0 (is Cesaro summable to 0). These criteria are applied to some classes of max--infinitely divisible processes.
We construct multiperiodic processes -- a simple example of stationary ergodic (but not mixing) processes over natural numbers that enjoy the vanishing entropy rate under a mild condition. Multiperiodic processes are supported on randomly…
Inspired by Hilberg's hypothesis, which states that mutual information between blocks for natural language grows like a power law, we seek for links between power-law growth rate of algorithmic mutual information and of some estimator of…
A regular Hilberg process is a stationary process that satisfies both a hyperlogarithmic growth of maximal repetition and a power-law growth of topological entropy, which are a kind of dual conditions. The hyperlogarithmic growth of maximal…
The data for many classification problems, such as pattern and speech recognition, follow mixture distributions. To quantify the optimum performance for classification tasks, the Shannon mutual information is a natural information-theoretic…
We propose a compression-based version of the empirical entropy of a finite string over a finite alphabet. Whereas previously one considers the naked entropy of (possibly higher order) Markov processes, we consider the sum of the…