Related papers: Empirical entropy in context
We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find…
It is well known that a Shannon based definition of information entropy leads in the classical case to the Boltzmann entropy. It is tempting to regard the Von Neumann entropy as the corresponding quantum mechanical definition. But the…
The goal of this paper is to develop an estimate for the entropy of random long-range correlated symbolic sequences with elements belonging to a finite alphabet. As a plausible model, we use the high-order additive stationary ergodic Markov…
Entropy estimation, due in part to its connection with mutual information, has seen considerable use in the study of time series data including causality detection and information flow. In many cases, the entropy is estimated using…
The weak law of large numbers implies that, under mild assumptions on the source, the Renyi entropy per produced symbol converges (in probability) towards the Shannon entropy rate. This paper quantifies the speed of this convergence for…
Thermodynamics can be formulated in either of two approaches, the phenomenological approach, which refers to the macroscopic properties of systems, and the statistical approach, which describes systems in terms of their microscopic…
Kolmogorov-Sinai entropy is an invariant of measure-preserving actions of the group of integers that is central to classification theory. There are two recently developed invariants, sofic entropy and Rokhlin entropy, that generalize…
The entropy of a binary symmetric Hidden Markov Process is calculated as an expansion in the noise parameter epsilon. We map the problem onto a one-dimensional Ising model in a large field of random signs and calculate the expansion…
We consider a number of proposals for the entropy of sets of classical coarse-grained histories based on the procedures of Jaynes, and prove a series of inequalities relating these measures. We then examine these as a function of the…
We survey the diverse approaches to the notion of information content: from Shannon entropy to Kolmogorov complexity. The main applications of Kolmogorov complexity are presented namely, the mathematical notion of randomness (which goes…
In this paper relations among some kinds of cumulative entropies and moments of order statistics are presented. By using some characterizations and the symmetry of a non negative and absolutely continuous random variable X, lower and upper…
The theory of ``Markov-up'' processes is being developed. This is a new class of stochastic processes with ``partial'' markovian features; it could also be called ``one-sided Markov''. Such a behavior may be found in the real world and in…
Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…
Numerical experiments support the interesting conjecture that statistical methods be applicable not only to fully-chaotic systems, but also at the edge of chaos by using Tsallis' generalizations of the standard exponential and entropy. In…
We study the behavior of a nonlinear semiclassical system using Shannon entropy and two approaches to statistical complexity. These systems involve the interaction between classical variables (representing the environment) and quantum ones.…
A possible mechanism of time is formulated by developing an idea of time replaced by quantum correlations, with the aid of modern quantum information theory. We invent a microscopic model, where correlations of a closed system are steadily…
We study the relations between the recently proposed machine-independent quantum complexity of P. Gacs~\cite{Gacs} and the entropy of classical and quantum systems. On one hand, by restricting Gacs complexity to ergodic classical dynamical…
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…
Uncertainty relations provide constraints on how well the outcomes of incompatible measurements can be predicted, and, as well as being fundamental to our understanding of quantum theory, they have practical applications such as for…
The Kolmogorov-Sinai entropy in the sense of Tsallis under Bernoulli shifts was obtained by Meson and Vericat [J. Math. Phys. 37, 4480(1996)]. In this paper, we propose a revised generalized Kolmogorov-Sinai-q entropy under Markov shifts.…